Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type @ followed by the MCP server name and your instructions, e.g., "@typesafe-as-a-judge rank these 3 test suites by relevance to the failing test"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
TypeSafe-as-a-Judge is a dual Codex and Claude Code plugin that gives coding agents a bounded semantic-judgment layer powered by TypeSafe Jev.
The agent remains responsible for orchestration, policy, permissions, code changes, and side effects. Jev supplies small typed signals—choices, scores, yes/no probabilities, and confidence—that the agent can use to route work, rank candidates, verify evidence, and decide when to ask for review.
It is deliberately not an autonomous authority layer.
Unofficial community integration: TypeSafe-as-a-Judge is not affiliated with, sponsored by, or endorsed by TypeSafe AI, OpenAI, Anthropic, Codex, or Claude. TypeSafe, Jev, OpenAI, Codex, Claude, and related names are the property of their respective owners.
Arik Aizikovich is currently the sole maintainer and contributor. Contributions are welcome: bug reports, documentation improvements, new examples, test cases, security feedback, and pull requests are all useful.
Please read CONTRIBUTING.md before opening a pull request. For security concerns, follow SECURITY.md instead of opening a public issue. The project uses Apache-2.0; contributions are accepted under the same license unless we explicitly agree otherwise.
Related MCP server: Jev MCP
Tool | Use it when | What it returns |
| One item must be sent to one known route | A route, probability distribution, confidence, and |
| Code has already retrieved a bounded candidate list | Ordered candidates with scores, probabilities, confidence, and a review recommendation |
| Code has already found possible values in a source | A selected candidate or |
| One claim needs to be checked against evidence | Support probability and a |
| Several independent narrow questions should run over shared state | Raw Choice, Score, and Noul signals for the caller's policy |
| A user asks how Jev was used or what it theoretically substituted | Current-process calls, tokens, elapsed time, per-tool breakdown, and declared substitute-model intent |
| Existing confidence or risk signals need one deterministic gate |
|
The tools are read-only. They do not edit files, approve actions, change records, invoke another model, or contact a third party.
The bundled skill tells Codex and Claude to consider this MCP when a task contains a bounded semantic decision. They do not call it for every request.
The normal pattern is:
Observe or retrieve the relevant state.
Reduce it to the minimum safe context.
Choose the narrowest TypeSafe tool.
Preserve the raw answer, probabilities, confidence, and threshold.
Proceed only when the explicit policy allows it.
Ask for review when confidence is low, no candidate fits, or evidence conflicts.
Coding-agent situation | Tool combination | Autonomous behavior |
An issue could be a bug, feature, documentation task, or investigation |
| Route the issue to one known workflow, or choose |
Several files may implement the requested behavior |
| Rank a retrieved shortlist by relevance before opening or editing candidates |
Several test suites could cover a change |
| Rank candidate suites using changed files, test names, and the requirement |
A failing test could be code, configuration, environment, dependency, or test failure |
| Classify the failure and choose the next diagnostic path |
Several implementation approaches are already known |
| Compare them against a concrete rubric such as compatibility, scope, and testability |
A natural-language request contains a known command, framework, or option |
| Select one value from candidates already found by code; return |
A generated summary makes claims about code or test results |
| Check each important claim against source evidence before presenting it as confirmed |
A citation or documentation link may not support a statement |
| Flag unsupported or partial evidence for review |
An extracted record contains multiple fields |
| Run independent field-level checks and keep each signal separately inspectable |
A plan has conflicting evidence or low-confidence choices |
| Escalate to the user or a stronger reasoning path instead of silently guessing |
A pull request has several possible review areas |
| Rank likely risk areas and verify claims about tests, behavior, and regressions |
A set of available skills could apply |
| Choose among a closed set of skills, with an explicit no-match path |
A task queue contains mixed work |
| Classify each item, then rank candidates within a defined category |
These are recommendations for the agent's next step. A recommendation is not permission to perform a consequential action.
The examples below show the shape of the MCP calls. Codex and Claude normally construct these arguments themselves when the task matches the skill.
Use typesafe_route when exactly one route should receive an item.
{
"state": {
"request": "I was charged twice and need a refund.",
"channel": "support"
},
"instructions": "Which team should handle this request?",
"routes": {
"billing": "Payments, invoices, refunds, and duplicate charges.",
"technical": "Bugs, outages, integrations, and broken product behavior.",
"account": "Login, identity, profile, and account-access issues."
},
"confidence_threshold": 0.8
}Possible result:
{
"route": "billing",
"confidence": 0.93,
"probabilities": {
"billing": 0.96,
"technical": 0.02,
"account": 0.01,
"needs_review": 0.01
},
"recommended_action": "proceed"
}The tool adds needs_review automatically. If the selected route is needs_review or confidence is below the threshold, the result is review.
Use typesafe_rank only after code has retrieved a bounded candidate list.
{
"instructions": "Which implementation is the best fit for adding tenant-scoped audit logging without widening authority?",
"context": {
"constraints": [
"Preserve existing RLS boundaries",
"Keep audit records append-only",
"Do not change authentication semantics"
]
},
"candidates": [
{ "id": "middleware", "value": "Add an audit middleware around protected mutations." },
{ "id": "orm-hook", "value": "Add a global ORM hook for all writes." },
{ "id": "db-trigger", "value": "Add database triggers to protected tables." }
],
"criteria": [
"Poor fit: violates a constraint or requires broad unsafe changes.",
"Good fit: satisfies the constraints with manageable integration work.",
"Excellent fit: satisfies the constraints, is auditable, and has a narrow blast radius."
],
"confidence_threshold": 0.7
}The agent receives an ordered list with a score and confidence for every candidate. It can inspect the top candidate, but should ask for review if the top result is not sufficiently distinct or confident.
Use typesafe_extract when deterministic code has already found possible values. This is selection, not free-form generation.
{
"source": "The deployment target is the EU production cluster in Frankfurt.",
"fields": [
{
"id": "deployment_target",
"instructions": "Which known deployment target is explicitly named by the source?",
"candidates": [
{ "id": "us_staging", "value": "us-staging", "description": "United States staging environment." },
{ "id": "eu_production", "value": "eu-production", "description": "European production environment." },
{ "id": "eu_staging", "value": "eu-staging", "description": "European staging environment." }
]
}
],
"confidence_threshold": 0.8
}If no supplied value is supported, the tool returns no_match and recommends review. It never creates a new environment name from the prose.
Use typesafe_verify for one specific claim and its evidence.
{
"claim": "The migration was replayed successfully on the evaluation database.",
"evidence": {
"command": "pnpm eval:db:fresh",
"exit_code": 0,
"output": "Applied 42 migrations; seed verification passed."
},
"support_threshold": 0.85
}Possible result:
{
"support_probability": 0.91,
"support_threshold": 0.85,
"recommended_action": "proceed"
}The result means the evidence supports the claim according to the question. It is not a deployment attestation, security approval, or substitute for the actual command output.
Use typesafe_judge when several narrow questions share the same state.
{
"state": {
"claim": "The release is ready for production.",
"evidence": {
"tests": "All focused tests passed.",
"migration": "Greenfield replay passed; production migration replay was not run.",
"approval": "No production approval is attached."
}
},
"questions": {
"tests_complete": {
"type": "noul",
"instructions": "Is the available test evidence sufficient for the stated release claim?",
"criteria": {
"true": "The relevant tests and checks are complete for the claim.",
"false": "Important tests or checks are missing."
}
},
"migration_evidence": {
"type": "choice",
"instructions": "What is the strongest migration evidence available?",
"criteria": {
"production_replay": "The intended production migration path was replayed.",
"greenfield_only": "Only a greenfield replay was performed.",
"none": "No meaningful migration evidence is present."
}
},
"approval_present": {
"type": "noul",
"instructions": "Is an explicit production approval present in the evidence?",
"criteria": {
"true": "The evidence contains an explicit approval.",
"false": "The evidence does not contain an explicit approval."
}
}
}
}The agent keeps each answer separate instead of collapsing the evidence into one vague overall score.
Use typesafe_escalation_gate after collecting model signals and deterministic checks.
{
"signals": [
{
"id": "route_confidence",
"value": 0.91,
"threshold": 0.8,
"comparator": "<=",
"reason": "Route confidence is too low."
},
{
"id": "citation_gap",
"value": 0.88,
"threshold": 0.8,
"comparator": ">=",
"reason": "A citation may not support the claim."
}
]
}Result:
{
"recommended_action": "review",
"fired_signals": [
{
"id": "citation_gap",
"value": 0.88,
"threshold": 0.8,
"comparator": ">=",
"reason": "A citation may not support the claim."
}
],
"gate": "max"
}The gate is deterministic. It does not decide what the reviewer should do and cannot execute the next action.
Use typesafe_usage_summary when a user asks how much Jev was used, how long it took, or what it theoretically saved. It records only metadata from Jev calls made by the current MCP server process: call count, input/output tokens, elapsed time, tool name, and an optional declared substitute model.
When a Jev call intentionally replaces a named model, include comparison_model in the call:
{
"comparison_model": "gpt-5.6-luna"
}That records substitution intent; it does not run Luna or assert a measured saving. A summary can then report that Jev handled a number of calls that were declared as alternatives to Luna, alongside the observed Jev tokens and elapsed time. It must say that token, time, and cost savings are theoretical unless a real baseline run exists.
The telemetry is process-local and resets when the MCP server restarts. It stores no request bodies, source text, credentials, or customer data.
Choice and Score answers include a confidence value derived from their probability distribution. Noul answers provide a probability that the statement is true; Noul does not provide a separate confidence value.
The agent should:
preserve the raw probability distribution and threshold in its work record;
treat needs_review and no_match as review outcomes;
escalate low-confidence results instead of silently selecting a winner;
use a max-style gate when one serious independent signal should be enough to trigger review;
calibrate thresholds against representative examples and the cost of a wrong decision;
keep useful uncertainty visible in the final explanation.
Confidence is not correctness. A high-confidence answer can still be wrong, and a low-confidence answer can reflect missing evidence rather than a false conclusion.
This example uses a realistic internal planning question that arises while extending this MCP. It is included to illustrate the decision shape, latency, and token accounting—not as a general performance benchmark.
User request: Add an MCP tool named typesafe_compare that compares two proposed implementation plans, returns an ordered score with a review recommendation, and adds tests and documentation.
Internal question: Which existing file should the coding agent inspect first?
Candidates:
server/judge.mjs - TypeSafe API client and existing tool handlers.
server/index.mjs - MCP tool schemas and JSON-RPC dispatch.
tests/judge.test.mjs - unit tests for tool behavior.
README.md - tool contracts and use cases.
Observation: The agent should locate the primary MCP integration surface before planning the implementation, tests, and documentation.
Measure | Ordinary reasoning | TypeSafe MCP judgment |
Model |
|
|
Result |
|
|
Elapsed time | 6.40 s | 0.861 s |
Agent usage | 13,236 input, 8,960 cached input, 52 output tokens | 1,174 input, 71 output tokens |
Price estimate | ~$0.00110 API-equivalent | At least ~$0.0000493 input-only |
Action |
|
|
Reason | It defines MCP schemas and JSON-RPC dispatch, so it is the first integration surface to inspect. | It ranked first with score 1.98 and confidence 0.98. |
Both paths selected the same first file and the same action. The TypeSafe path adds a reusable score, probability distribution, and confidence signal that a deterministic gate can consume.
The Luna estimate uses the current API rates of $0.20 per million uncached input tokens, $0.02 per million cached input tokens, and $1.20 per million output tokens. It is an API-equivalent calculation; Codex subscription billing can differ. See the GPT-5.6 Luna model page.
The TypeSafe estimate uses the published $42 per billion input tokens: 1,174 x $42 / 1,000,000,000 = $0.000049308. TypeSafe's public figure did not show output-token pricing, so this is a lower bound. See TypeSafe AI.
The two runs answer the same internal question, but they are not a controlled benchmark. The left side is an end-to-end Codex run; the right side is a direct MCP ranking call. Their token accounting measures different layers.
Do not use this MCP as the authority for:
authorization, authentication, RLS, tenant isolation, or permission grants;
financial calculations, payments, billing, or account changes;
security approval or vulnerability disposition;
production deployment, migration completion, or release approval;
destructive actions or irreversible external operations;
exact arithmetic, dates, identifiers, or deterministic lookups;
open-ended writing or general reasoning that does not need a typed judgment.
Those decisions belong to deterministic code, existing governance, explicit evidence, and human approval gates.
Send only the minimum state needed for one judgment. Do not send:
API keys, passwords, access tokens, or private keys;
payment data or secrets from configuration files;
unrelated tenant or company data;
unredacted sensitive personal information;
large repositories when a small excerpt or candidate list is sufficient.
The plugin returns TypeSafe usage metadata, but it does not persist a semantic decision history. Callers should decide what to retain in their own auditable work records.
codex plugin marketplace add E-FL/typesafe-as-a-judge
codex plugin add typesafe-as-a-judge@typesafe-as-a-judgeThen configure the TypeSafe token from a checkout of this repository:
git clone https://github.com/E-FL/typesafe-as-a-judge.git
cd typesafe-as-a-judge
powershell -ExecutionPolicy Bypass -File .\scripts\setup.ps1The Windows setup prompts without echoing the token, validates it with a harmless Jev request, and stores it in the Windows user environment. Restart Codex and start a new task after setup.
claude plugin marketplace add E-FL/typesafe-as-a-judge
claude plugin install typesafe-as-a-judge@typesafe-as-a-judgeRun the same guided setup from the repository checkout. Restart Claude Code and approve the project/plugin MCP server when prompted.
git clone https://github.com/E-FL/typesafe-as-a-judge.git
cd typesafe-as-a-judge
chmod +x ./scripts/setup.sh
./scripts/setup.shThe Unix setup stores the token at ~/.config/typesafe-as-a-judge/token with owner-only permissions. The MCP launcher reads that file when TYPESAFE_API_KEY is not already present.
Never paste a TypeSafe token into Codex or Claude chat, commit it to a file, or place it in MCP command-line arguments.
This plugin is a local STDIO MCP server. Codex and Claude launch it as a local process and pass TYPESAFE_API_KEY through the session environment. It does not use an OAuth login button or a hosted credential broker.
The TypeSafe request is:
POST https://api.typesafe.ai/v1/systemone
Authorization: Bearer <TYPESAFE_API_KEY>
model: jev-latestThe token is never included in the plugin package or repository.
Node.js 20 or later is required.
Requests are limited to 256 KiB before being sent.
Transient TypeSafe 429 and 529 responses receive bounded exponential backoff.
The server returns TypeSafe model and usage metadata with judgments.
typesafe_rank accepts at most 50 candidates.
typesafe_extract accepts at most 20 fields and 100 candidates per field.
typesafe_judge accepts at most 25 independent questions.
typesafe_escalation_gate accepts at most 100 signals.
The MCP server writes protocol messages to stdout; diagnostics must not be written to stdout.
Install no runtime dependency: the MCP server uses Node.js built-ins so a fresh marketplace install works without node_modules.
Run the local tests:
npm testThe test suite covers tool behavior, missing-token failure, the launcher, and the JSON-RPC MCP handshake without making a live TypeSafe request.
Validate the Codex and Claude manifests:
py -3 "$env:USERPROFILE\.codex\skills\.system\plugin-creator\scripts\validate_plugin.py" (Get-Location)
claude plugin validate .For direct MCP debugging:
npm startThe process communicates over standard input/output. Do not write normal logs to stdout.
Apache-2.0. See LICENSE.
Apply a deterministic max-style review gate to explicit probabilities or confidences. No model call, writes, or external action occurs.
| Name | Required | Description | Default |
|---|---|---|---|
| signals | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false. The description adds 'No model call, writes, or external action occurs' and 'deterministic', but these largely reinforce the annotations. The phrase 'max-style review gate' is vague and does not explain the actual gating logic, thresholds, or return value. With no output schema, the agent is left without critical behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise—two short sentences with no fluff. The core action is front-loaded in the first sentence, and the second sentence adds a safety qualifier. It is appropriately sized, though the brevity contributes to under-specification.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool takes a complex array input with nested objects and no output schema, yet the description does not explain the decision logic, what the tool returns (e.g., a boolean, list of triggered signals), or the meaning of 'max-style'. An agent cannot reliably infer how to invoke this tool correctly or interpret its result.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It only says 'explicit probabilities or confidences' without explaining that the 'signals' parameter is an array of objects with id, value, threshold, and comparator fields. The structure and semantics of the only parameter are left entirely to the schema, which lacks prose descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('Apply') and resource ('deterministic max-style review gate') and clarifies the input domain ('explicit probabilities or confidences'). It also distinguishes itself from model-calling tools by noting 'No model call, writes, or external action occurs.' However, it does not explicitly name or contrast sibling tools, so the differentiation is implicit rather than explicit.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to use this tool versus the sibling tools (typesafe_route, typesafe_rank, etc.). No conditions, exclusions, or alternatives are mentioned. An agent is left to infer usage context from the name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Select supported values from candidates code already found in a source. It cannot generate values. Returns selected values, confidence, and review fields.
| Name | Required | Description | Default |
|---|---|---|---|
| fields | Yes | ||
| source | Yes | Source text or structured source to evaluate. | |
| comparison_model | No | Optional model this call intentionally substitutes. Recorded only for a theoretical usage summary; no baseline is run. | |
| confidence_threshold | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly, openWorld, idempotent, and non-destructive, so the description only needs to add behavioral nuance. It adds that the tool cannot generate values and returns selected values, confidence, and review fields, which are useful behavioral details. No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences with no extraneous words. The key constraint (cannot generate) is front-loaded in the second sentence, and the output summary is concise. It is well-structured for quick parsing.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has four parameters, two required, and no output schema. The description gives a high-level purpose but omits details on how to specify fields and candidates, the meaning of confidence_threshold, and the exact shape of the return. An agent would need to rely on the schema, which itself is incomplete for two parameters, making the overall context insufficient for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema covers source and comparison_model with descriptions, but fields and confidence_threshold lack descriptions. The description does not add any parameter-specific meaning, such as how to structure the fields array or what confidence_threshold controls. With only 50% schema coverage, the description should compensate but does not.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function: selecting supported values from pre-existing candidates, and explicitly notes it cannot generate values. This distinguishes it from generation tasks, though it doesn't explicitly contrast with sibling tools like typesafe_rank or typesafe_verify. The verb 'select' and resource 'candidates' are specific enough to convey purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when candidates already exist and need selection, and explicitly says it cannot generate values, suggesting it should not be used for generation. However, it does not explicitly name alternatives or conditions for when to use this tool over siblings. The guidance is inferred rather than stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Run up to 25 independent, narrow TypeSafe Choice, Score, or Noul questions against shared state. Returns raw typed signals for the caller's explicit policy.
| Name | Required | Description | Default |
|---|---|---|---|
| state | Yes | Minimum relevant JSON state. | |
| questions | Yes | Question id to a TypeSafe Choice, Score, or Noul question. | |
| comparison_model | No | Optional model this call intentionally substitutes. Recorded only for a theoretical usage summary; no baseline is run. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already establish read-only, idempotent, non-destructive behavior, so the bar is lower. The description adds useful behavioral context beyond those annotations: a 25-question cap, independence of questions, shared state, and a raw-signal output style. It does not contradict the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences with no filler. The first sentence front-loads the action, limit, question types, and shared-state scope; the second clarifies the output style. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with nested object parameters and no output schema, the description covers the core behaviors an agent needs: what it runs, against what state, at what scale, and what kind of result it returns. It could be more complete about the shape of the returned signals or the question format, but the annotations and high schema coverage carry much of the remaining burden.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3, but the description adds meaning not present in the schema: the 'up to 25' limit and the 'independent, narrow' constraint directly qualify the questions parameter. It does not add detail about state or comparison_model, but those are already adequately described in the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific action ('Run'), a specific resource ('TypeSafe Choice, Score, or Noul questions'), and a clear scope ('against shared state'). It also says what the tool returns ('raw typed signals'), but it does not explicitly compare itself to sibling tools such as typesafe_route or typesafe_rank, so it stops short of full differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The phrase 'up to 25 independent, narrow ... questions' clearly implies a batch-judgement use case, and 'for the caller's explicit policy' suggests the tool defers decision-making to the caller. However, no sibling alternatives are named, and there is no explicit statement of when not to use this tool, leaving routing largely to inference.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Score and rank up to 50 retrieved candidates against a concrete rubric. Returns scores, confidence, and a review recommendation; it never selects or changes a record.
| Name | Required | Description | Default |
|---|---|---|---|
| context | No | Optional minimum relevant JSON context. | |
| criteria | Yes | ||
| candidates | Yes | ||
| instructions | Yes | What makes a candidate good for this task. | |
| comparison_model | No | Optional model this call intentionally substitutes. Recorded only for a theoretical usage summary; no baseline is run. | |
| confidence_threshold | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, openWorldHint, idempotentHint, and non-destructive behavior. The description adds value by stating the tool returns scores, confidence, and a review recommendation, and by explicitly reinforcing that it never selects or changes a record. This goes beyond the annotations without contradicting them.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences with no filler. The core action is front-loaded, and the second sentence efficiently conveys output and a key limitation. Every clause earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with six parameters, no output schema, and sibling tools, the description covers the essential invocation context: what to provide, what will be returned, and what the tool will not do. It omits optional parameters' behavior, but the annotations and schema fill some of that gap, so the definition is reasonably complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 50%, so the description needs to compensate for undocumented parameters. It adds meaning for 'candidates' (retrieved, up to 50) and 'criteria' (concrete rubric), but it does not clarify 'confidence_threshold', 'comparison_model', or 'instructions' beyond what the schema already states. The description helps but does not fully bridge the gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Score and rank'), identifies the resource ('retrieved candidates'), and bounds the scope ('up to 50') against 'a concrete rubric'. It also distinguishes itself from sibling tools by stating it never selects or changes a record, which separates it from routing and mutation tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives clear context: use this when you have retrieved candidates that need scoring and ranking against a rubric. It also provides an exclusion ('it never selects or changes a record'), but it does not explicitly name sibling alternatives or conditions for when to use them instead.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Choose one route from a closed set using TypeSafe Jev. Returns a route plus a confidence-aware proceed/review recommendation; it never performs the route.
| Name | Required | Description | Default |
|---|---|---|---|
| state | Yes | Minimum relevant JSON state. | |
| routes | Yes | Route id to rubric description. The tool adds needs_review. | |
| instructions | Yes | One narrow routing question. | |
| review_route | No | Optional reserved no-match route id; defaults to needs_review. | |
| comparison_model | No | Optional model this call intentionally substitutes. Recorded only for a theoretical usage summary; no baseline is run. | |
| confidence_threshold | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, openWorldHint, idempotentHint, and non-destructive. The description adds that it 'never performs the route' and returns a 'confidence-aware proceed/review recommendation,' which provides concrete behavioral context beyond the annotations. This is consistent and adds value, though it does not cover edge cases like error handling or side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise, two sentences with no fluff. The primary action is front-loaded ('Choose one route'), and the key behavioral caveat ('never performs the route') is included. Every sentence contributes to understanding the tool's purpose and limitations.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has 6 parameters, nested objects, and no output schema. The description gives a high-level overview but does not detail the output structure beyond mentioning a route and a recommendation. While the schema covers parameters well, the lack of output schema and minimal description of return format leaves some ambiguity. The description is sufficient for basic usage but not fully comprehensive.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 83%, so most parameters are already documented. The description adds minimal parameter context, such as 'closed set' for routes and the notion that routes are not executed, which helps interpret the routes parameter. However, it does not explain the role of confidence_threshold, review_route, or comparison_model beyond what the schema provides. This is adequate given high coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's action ('Choose one route'), the resource ('from a closed set'), and explicitly notes it never performs the route. This distinguishes it from siblings like typesafe_rank or typesafe_extract, which have different purposes. The verb and resource are specific and unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the tool is for selecting among predefined routes and explicitly states it never performs the route, which hints at non-execution usage. However, it does not name any sibling tools or provide explicit when-to-use vs when-not-to-use guidance. The context is clear but not fully developed.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Summarize Jev calls made by the current MCP server process: tool count, token usage, elapsed time, and declared substitute-model intent. It never runs a baseline and labels savings as theoretical.
| Name | Required | Description | Default |
|---|---|---|---|
| comparison_model | No | Optional declared substitute model to filter the current-process summary. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the annotations declaring readOnly, idempotent, and non-destructive behavior, the description adds crucial behavioral disclosure: it never runs a baseline and labels savings as theoretical. This prevents an agent from over-interpreting the numbers and adds real value beyond what annotations provide.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with no filler. The first sentence front-loads what the tool does and what it returns, and the second sentence adds a key caveat about baseline behavior and theoretical savings. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple read-only summary tool with one optional parameter and no output schema, the description is complete. It identifies the scope, the data fields returned, the caveat about theoretical savings, and the fact that no baseline is run. An agent has enough information to call and interpret the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The single optional parameter is fully documented in the schema, with schema description coverage at 100%. The parameter description in the schema already explains that comparison_model filters the current-process summary, so the tool description does not need to repeat it. Baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Summarize') with a clear resource ('Jev calls made by the current MCP server process') and enumerates the exact content: tool count, token usage, elapsed time, and substitute-model intent. This clearly distinguishes it from the action-oriented sibling tools like typesafe_route and typesafe_verify.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description makes clear this is a process-level usage summary rather than an operational tool, which effectively indicates when it applies. It does not explicitly state when not to use it or name alternatives, but the sibling tools are sufficiently distinct that the usage context is unambiguous.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Judge whether supplied evidence directly supports a claim. Returns a support probability and an explicit proceed/review recommendation; it never treats a claim as proven on its own.
| Name | Required | Description | Default |
|---|---|---|---|
| claim | Yes | ||
| context | No | Optional relevant JSON context. | |
| evidence | Yes | The source passage(s) or structured evidence. | |
| comparison_model | No | Optional model this call intentionally substitutes. Recorded only for a theoretical usage summary; no baseline is run. | |
| support_threshold | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, openWorldHint, idempotentHint, and destructiveHint. The description adds behavioral nuance beyond these: it never treats a claim as proven on its own and returns a proceed/review recommendation. This enriches the agent's understanding without contradiction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences with zero fluff. The purpose is front-loaded, and the second sentence adds essential behavioral detail. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with no output schema, the description states the return types (support probability and recommendation) but omits how to interpret them or how support_threshold influences the outcome. It is adequate for basic use but leaves some operational details unexplained.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 60%, so the description must partially compensate. It clarifies the role of claim and evidence through its purpose statement, but it does not explain support_threshold or comparison_model beyond what the schema already provides. It adds some value but does not fully fill the gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool judges whether evidence supports a claim, which is a specific verb-resource pair. It adds details about the return (support probability and recommendation) but does not explicitly distinguish it from sibling tools like typesafe_judge, so it stops short of a 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is given on when to use this tool versus siblings such as typesafe_judge or typesafe_route. The purpose implies usage but lacks explicit exclusions or alternative routing conditions, leaving the agent to infer applicability.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
v0.1.0typesafe_escalation_gatetypesafe_extracttypesafe_judgetypesafe_ranktypesafe_routetypesafe_usage_summarytypesafe_verifyScored across 7 tools
Most tools are individually clear, but typesafe_judge can execute Choice, Score, and Noul questions, which conceptually overlaps with route, rank, extract, and verify. The descriptions provide enough context to help avoid misselection, but the boundaries are not always crisp.
All tools share a uniform typesafe_ prefix and snake_case style, making the family recognizable. The majority use verb-style names like route, rank, extract, verify, and judge, though usage_summary and escalation_gate are noun-style deviations.
Seven tools is well-scoped for a specialized judging and evaluation server. Each tool addresses a distinct phase or concern without feeling padded or redundant.
The surface covers the core judge lifecycle: route, rank, extract, verify, batch judging, usage tracking, and escalation gating. Minor gaps such as a dedicated audit or explanation tool are possible, but they are not obvious blockers for the server's stated purpose.
Pre-execution governance for AI agents. Deterministic PASS/FAIL/REVIEW verdicts, replayable proof.
Human judgment for AI agents: discover capabilities, get quotes, and track paid human tasks.
Governance layer for AI coding agents: knowledge-graph grounding, session audit, policy controls.
Trust signals for AI agents: an open agent-readiness standard and developer tool guide. Read-only.
