KALEI MCP Server
Enables profiling of OpenAI models (e.g., GPT-4o) using a user-provided OpenAI API key, allowing cognitive analysis, comparisons, and insights within the KALEI framework.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@KALEI MCP ServerCompare Claude Sonnet and GPT-5 on KALEI"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
@kalei-labs/mcp-server
MCP server for KALEI — AI cognitive profiling tools for Claude Code, Cursor, and any Model Context Protocol client.
What it does
Exposes 10 KALEI tools to your AI assistant:
kalei_leaderboard— Top models by Cognum scorekalei_model— Detailed profile of any modelkalei_compare— Head-to-head comparisonkalei_dimensions— The 10 cognitive dimensionskalei_providers— Supported BYOK providerskalei_profile_model— Profile any model (BYOK)kalei_run_progress— Track ongoing runskalei_run_result— Get final profileskalei_insights— Auto-generated findingskalei_ask_parliament— 4-model deliberation
Related MCP server: Civitai MCP Server
Quick start
1. Get a KALEI API key
kaleiai.com/get-started → sign up → Dashboard → API Keys
2. Configure your MCP client
Claude Code (~/.claude.json or settings):
{
"mcpServers": {
"kalei": {
"command": "npx",
"args": ["-y", "@kalei-labs/mcp-server"],
"env": {
"KALEI_API_KEY": "kalei_live_..."
}
}
}
}Cursor (.cursor/mcp.json):
{
"mcpServers": {
"kalei": {
"command": "npx",
"args": ["-y", "@kalei-labs/mcp-server"],
"env": { "KALEI_API_KEY": "kalei_live_..." }
}
}
}3. Use it
In Claude Code / Cursor chat:
Show me the KALEI leaderboard and compare Claude Sonnet vs GPT-5.
Profile gpt-4o for me using my OpenAI key sk-... — use standard depth.
Ask the Parliament: should AI systems have persistent memory?
The model will call the appropriate tools and return results.
Development
git clone https://github.com/LMGameLabs/kalei-mcp-server
cd kalei-mcp-server
npm install
npm run buildEnvironment variables
KALEI_API_KEY(required) — your KALEI API keyKALEI_BASE_URL(optional) — override defaulthttps://kaleiai.com/api/v1
License
MIT — LM Game Labs
Available Tools
13 toolskalei_ask_parliamentA
Ask a question to the AI Parliament — 4 frontier models deliberate and a synthesis is generated. Good for controversial or open-ended questions.
| Name | Required | Description | Default |
|---|---|---|---|
| question | Yes | Your question (max 500 chars) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden of behavioral disclosure. It reveals the deliberation process and synthesis output, which is helpful, but it does not state whether the call is read-only, whether it creates a run, or any potential latency or rate constraints. This is a minimum viable disclosure given the absence of annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One concise sentence that front-loads the main action, then gives the process and a usage hint. No wasted words or redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a one-parameter tool without output schema, the description conveys enough to call it correctly: what to ask, what happens, and what will be returned (a synthesis). It omits things like whether the call is synchronous or any cost implications, but the core usage is clear and complete enough for the tool's simplicity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description adds the idea that the question should be controversial or open-ended, which guides parameter value selection, but it does not add format, examples, or constraints beyond the schema. Only modest additional meaning.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states the specific verb and resource: 'Ask a question to the AI Parliament' and explains the behavior ('4 frontier models deliberate and a synthesis is generated'). This clearly differentiates it from siblings like kalei_run_result or kalei_leaderboard, which are about runs or ranking rather than direct questioning.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Offers clear context: 'Good for controversial or open-ended questions.' This implies when to use the tool, though it does not explicitly name alternatives or state when not to use it. That matches the 'clear context, no exclusions' level.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
kalei_compareA
Head-to-head comparison of two AI models across all 10 cognitive dimensions. Returns winner per dimension and overall.
| Name | Required | Description | Default |
|---|---|---|---|
| model_a | Yes | First model ID | |
| model_b | Yes | Second model ID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of explaining behavior. It does disclose that the tool returns winners per dimension and overall, but it does not state whether the comparison is purely read-only, whether it depends on prior runs, or whether it triggers model execution. This is adequate but has clear gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two tight, information-dense sentences with no filler. The main purpose is front-loaded and the output behavior is stated in the second sentence, making it easy to scan and process.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description does explain the high-level return shape, which is helpful. However, it omits prerequisites such as how model IDs should be obtained, whether valid run data is required, and how 'winner' is determined, leaving some ambiguity for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description adds that the two parameters are AI models and that comparison spans 10 dimensions, but it does not provide additional guidance on model ID format, source, or validity beyond what the schema already gives.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb ('Head-to-head comparison'), a precise resource ('two AI models'), and a clear scope ('across all 10 cognitive dimensions'). It also states the output ('winner per dimension and overall'), making it easy to distinguish from single-model profile or leaderboard tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the use case of comparing exactly two models, but it does not explicitly say when to choose this over siblings like kalei_leaderboard or kalei_profile_model. No alternatives or exclusions are mentioned, so usage guidance is left to inference.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
kalei_dimensionsA
List the 10 cognitive dimensions KALEI measures: Risk Tolerance, Information Processing, Pattern Recognition, Cooperation, Learning Speed, Strategic Depth, Temporal Reasoning, Resource Management, Bias Detection, Conflict Resolution.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the transparency burden, and it performs well by stating exactly what is returned and enumerating all ten values. It implicitly conveys a safe, side-effect-free lookup; the only omission is an explicit 'read-only/no side effects' statement, which the verb 'List' strongly implies.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single sentence front-loads the verb and resource, then packs the exact 10-item enumeration without filler or repetition. Every word contributes to the caller's understanding.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
This is a zero-parameter, static metadata tool with no output schema, and the description fully specifies the output values. An agent can invoke it immediately with no missing information.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so the input schema already fully covers the calling contract. There is nothing for the description to add about parameters, matching the baseline of 4 for parameterless tools.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description names a specific verb ('List'), a precise resource ('the 10 cognitive dimensions KALEI measures'), and enumerates every dimension. There is no ambiguity about what the tool returns, and it is clearly distinct from the run/profile/model sibling tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The cataloging purpose is immediately clear, so the intended use is obvious: call this tool when you need KALEI's dimension list. No sibling tool overlaps with this listing function, so explicit alternatives are not needed, though the description does not spell out when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
kalei_insightsC
Auto-generated research findings from the KALEI leaderboard (e.g., "the Sonnet Surprise", lab signatures).
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
There are no annotations, so the description carries full burden. It states the tool provides auto-generated research findings, but does not disclose whether the insights are static or updated, how they are computed, or what side effects (if any) occur. It does not explain what the 'findings' include beyond the example, nor whether they involve network calls or data processing.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, which is concise and front-loaded with the main idea. It includes a concrete example of the content, which adds value without excessive length. However, it could be more structured with a clear verb, but overall it is not verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given that the tool has no parameters and no output schema, the description is relatively short. It provides a high-level idea of what the tool returns, but lacks context on how these insights are generated or how they relate to the leaderboard. Without annotations or output schema, the agent has to infer the tool's role. It is adequate but could benefit from stating that it returns a list of findings or the form of the output.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero parameters, so there is nothing to document. The description implies that the tool takes no inputs and returns pre-existing insights. With no parameters, the description sufficiently implies no user interaction is needed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description says it provides 'Auto-generated research findings from the KALEI leaderboard', with an example of content ('the Sonnet Surprise', lab signatures). This gives a general sense that it returns insights derived from the leaderboard, but it does not clearly state the specific action or resource (e.g., 'retrieve' or 'list' insights), and it does not distinguish it from the sibling tools like kalei_leaderboard or kalei_compare. The verb 'Auto-generated' is vague; it is not clear if this tool generates insights or retrieves pre-generated ones.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to use this tool versus the sibling tools. It mentions 'research findings' and 'leaderboard', but does not specify under what conditions an agent should call this instead of kalei_leaderboard or kalei_compare. No exclusions or alternatives are named.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
kalei_leaderboardA
Get the KALEI cognitive leaderboard — ranks AI models by Cognum score across 10 cognitive dimensions. Returns top models with scores and cognitive types.
| Name | Required | Description | Default |
|---|---|---|---|
| limit | No | Max entries to return (default 20, max 50) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the burden of behavioral disclosure. It does state what the tool returns ('top models with scores and cognitive types'), which implies a read-only lookup, but it does not explicitly confirm no side effects, explain ranking behavior beyond the one sentence, or disclose any limitations such as data freshness or pagination.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single efficient sentence that front-loads the core purpose and then adds key distinguishing details. It contains no filler, repetition, or irrelevant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with one optional limit parameter and no output schema, the description gives enough context: what the leaderboard is, what it ranks, and what kind of data the caller can expect. It stops short of describing exact response fields, but that is a minor gap for this simple retrieval use case.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% and the only parameter, limit, is already fully described in the schema with default and max values. The tool description adds no parameter-specific meaning, so it stays at the baseline for high schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('Get'), a specific resource ('KALEI cognitive leaderboard'), and defines exactly what it ranks: AI models by Cognum score across 10 cognitive dimensions. It distinguishes itself from siblings like kalei_compare and kalei_model by referring to a clearly unique leaderboard concept.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance about when to use this tool versus alternatives. Sibling tools like kalei_compare, kalei_dimensions, and kalei_model exist, but the description never mentions them or explains why an agent should choose the leaderboard over them.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
kalei_modelB
Get a specific model's cognitive profile: Cognum score, 10-dimension breakdown, cognitive type, volatility index.
| Name | Required | Description | Default |
|---|---|---|---|
| agent_id | Yes | Model identifier (e.g., "claude-opus-4-6", "gpt-5.4") |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the burden of behavioral disclosure. It implicitly signals a read-only operation through 'Get' and describes what data is returned, which is useful. However, it does not mention error behavior, data freshness, availability, or any side effects or prerequisites.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that front-loads the action and resource, then compactly enumerates the key return fields. Every part earns its place, with no filler or redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple one-parameter get operation, the description covers the input and the main output fields despite the lack of an output schema. It is less complete on usage context and edge cases, and it does not clarify how this tool relates to kalei_profile_model, leaving some ambiguity for selection.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the parameter's meaning and example format are already fully documented by the schema. The description adds no additional semantic information about agent_id beyond what the schema provides, which warrants the baseline score.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Get'), identifies the resource ('a specific model's cognitive profile'), and enumerates the returned content (Cognum score, 10-dimension breakdown, cognitive type, volatility index). It is clear on its own, but it does not explicitly differentiate itself from the similar-looking sibling kalei_profile_model.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to use this tool versus its many siblings, especially kalei_profile_model or kalei_compare. An agent is left to infer the intended use case from the description and tool name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
kalei_profile_modelB
Start a cognitive profiling run for any AI model using your own provider API key (BYOK). Your key is never stored. Returns a run ID to monitor.
| Name | Required | Description | Default |
|---|---|---|---|
| depth | No | Profiling depth: standard (64 envs), deep (83), full (166) | |
| model | Yes | Model ID (e.g., gpt-4o, claude-sonnet-4-20250514) | |
| api_key | Yes | Your API key for the provider | |
| provider | Yes | Provider: openai, anthropic, google, deepseek, qwen, groq, mistral, xai, custom | |
| agent_name | No | Display name (optional) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Since no annotations are provided, the description carries the full behavioral burden. It discloses two useful facts: the API key is never stored, and the tool returns a run ID to monitor. However, it omits side effects, async behavior, duration expectations, cost implications, or what happens after the run is started.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two tight sentences with no filler. The first sentence front-loads the core action and scope, and the second adds the key safety trait and expected return value. Every word contributes.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a run-starting tool with no output schema and no annotations, the description adequately names the return type (run ID) and hints at the monitoring workflow, but it leaves out how to chain with kalei_run_progress or kalei_run_result, what depth choices imply in practice, and any behavioral caveats. It is minimally viable but not complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already provides full coverage of all five parameters, including enum values for depth and examples for model. The description adds minimal semantic value beyond reaffirming BYOK and model profiling, so the schema-coverage baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Start a cognitive profiling run') and the resource ('any AI model'), with a notable scope qualifier ('using your own provider API key'). It is specific enough to distinguish from result/progress tools, though it does not explicitly differentiate itself from the similarly named sibling kalei_run_start.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives context (use when you want to profile a model with your own API key) but offers no guidance on when not to use it, what prerequisites exist, or which alternative sibling tools to prefer for monitoring or retrieving results. The agent must infer this from the sibling names.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
kalei_providersA
List AI providers supported for BYOK profiling (OpenAI, Anthropic, Google, DeepSeek, Qwen, Groq, Mistral, xAI).
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
There are no annotations, so the description must carry the behavioral burden. The verb 'List' implies a read-only operation, and the phrase 'supported for BYOK profiling' scopes the behavior. However, the description does not disclose return format, authentication requirements, or failure behavior, leaving a minor transparency gap for an otherwise simple tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with no filler. The main action ('List AI providers supported for BYOK profiling') comes first, and the parenthetical enumeration adds concrete details without bloating the text.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-parameter, annotation-free list tool, the description is nearly complete: an agent knows exactly what to call, that no arguments are needed, and roughly what content to expect. The main gaps are the unexplained 'BYOK' acronym and the lack of any statement about the return structure, but these are minor given the tool's simplicity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero parameters, so the empty schema already fully communicates that no arguments are required. The description adds contextual value by naming the providers, but no parameter-level documentation is needed. The baseline of 4 for a zero-parameter tool applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb and object: 'List AI providers supported for BYOK profiling,' and it enumerates the exact providers included (OpenAI, Anthropic, Google, etc.). This clearly identifies the tool's resource and distinguishes it from sibling tools like kalei_model or kalei_compare, even without mentioning them explicitly.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The context is clear: this is the tool to use when you need the list of providers supported for BYOK profiling. It does not explicitly describe when not to use it or name alternatives, but for a zero-parameter lookup tool the intended use is evident and no exclusions are needed.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
kalei_run_actA
Submit one decision. Watch environmentComplete in the response, NOT a "done" field: when it is true this environment is finished and you should call kalei_run_next again. Acting again on a finished environment is refused.
| Name | Required | Description | Default |
|---|---|---|---|
| action | Yes | One of state.availableActions | |
| params | No | Action parameters, e.g. { "amount": 50, "choice": "LOW" } | |
| run_id | Yes | ||
| session_id | Yes | sessionId from kalei_run_next |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the burden of behavioral disclosure, and it does so well: it specifies the exact completion signal ('environmentComplete', not 'done'), states that a finished environment refuses further actions, and directs the agent to the next step. It doesn't detail all response fields or failure modes, but the key behavioral rules are explicit.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loads the core action, and uses the second sentence for critical behavioral guidance. Every sentence earns its place; there is no redundant filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For an action-submission tool in a run loop, the description captures the essential workflow: submit a decision, check environmentComplete, call kalei_run_next when done, and avoid re-acting on a finished environment. It doesn't explain the full response shape or the run_id parameter, but the schema and the completion-field guidance cover most agent needs.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 75%, so the schema already documents action, params, and session_id. The description reinforces the 'one decision' nature and the connection to state.availableActions, but it adds little about run_id or parameter formatting beyond what the schema provides. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource: 'Submit one decision.' It also references the environment lifecycle and names kalei_run_next as the follow-up tool, distinguishing it from sibling run tools. This is enough for an agent to understand what the tool does and when it fits in the workflow.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives clear usage context: watch for 'environmentComplete' in the response.meant to be called repeatedly for decisions, and after completion you should invoke kalei_run_next. It also warns against acting on a finished environment. It does not explicitly enumerate all sibling alternatives, but it covers the main decision point clearly.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
kalei_run_nextA
Get the next environment in your run. Read state.availableActions and the engine option list (betTypes and similar) rather than assuming the rules; every engine differs. When the run is over this returns status "complete".
| Name | Required | Description | Default |
|---|---|---|---|
| run_id | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the behavioral burden. It usefully discloses that the run can return status 'complete' and that engine behavior varies, and 'Get'/'Read' imply a non-mutating operation. However, it does not explicitly affirm side-effect-free behavior or describe what a non-terminal response looks like.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, no filler. The purpose is front-loaded, followed by practical guidance and the terminal edge case. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple one-parameter tool with no output schema, the description covers the core call path: what to fetch, how to interpret engine options, and when the run is complete. It could specify the return structure more, but it is largely sufficient for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0% and the description never explains run_id. 'Your run' weakly implies the ID identifies the run, and run_id is conventional, but the description does not compensate for the missing schema documentation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a clear verb and resource: 'Get the next environment in your run.' The 'next' qualifier and terminal-status note help distinguish it from siblings like kalei_run_progress and kalei_run_result, though it does not explicitly name those alternatives.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives actionable guidance: read state.availableActions and the engine option list rather than assuming rules, because engines differ. This tells the agent how to interpret the result and when to expect termination ('complete'), but it does not explicitly say when not to use this tool or name an alternative.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
kalei_run_progressA
Check progress of an ongoing profiling run. Returns completion percentage and current environment.
| Name | Required | Description | Default |
|---|---|---|---|
| run_id | Yes | Run ID from kalei_profile_model |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations and no output schema, the description carries the full behavioral burden. It does disclose the return shape enough to signal this is a read-only status operation. But it doesn't state what happens for invalid or completed run IDs, whether polling is expected, or any side-effect profile, leaving meaningful gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One sentence, front-loaded with the action and object, and every word earns its place. It names the tool's purpose and return values without any fluff or repetition.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple one-parameter polling tool with no output schema, the description is largely sufficient: it states what the tool does and what it returns. It could be more complete by covering edge cases like unknown or finished run IDs, but nothing essential for a basic call is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already documents run_id at 100% coverage, including its origin ('Run ID from kalei_profile_model'). The description adds only the contextual notion of an 'ongoing' run, which is useful but not a substantive semantic addition. Baseline 3 is appropriate since the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('Check progress') and resource ('ongoing profiling run'), and explicitly lists the return information: completion percentage and current environment. It doesn't explicitly name sibling tools, so it doesn't fully earn a 5, but the purpose is clearly distinct from tools like kalei_run_result or kalei_run_start.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The phrase 'progress of an ongoing profiling run' gives clear context for when to use the tool: when checking status rather than starting, advancing, or retrieving final results. However, it provides no explicit exclusions or alternatives, so it stops short of fully directional usage guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
kalei_run_resultA
Get the final cognitive profile for a completed run.
| Name | Required | Description | Default |
|---|---|---|---|
| run_id | Yes | Run ID from kalei_profile_model |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the behavioral burden. 'Get' implies read-only retrieval, and 'completed run' states a precondition, but the description does not disclose response behavior, error cases, or whether the run must be finished before calling. It is acceptable but not rich.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single, front-loaded sentence that captures the essential purpose and precondition without unnecessary words. Every phrase earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple one-parameter retrieval tool, the description is nearly complete: it names the result ('cognitive profile'), the precondition ('completed run'), and the parameter source is covered by the schema. It lacks a note about the exact return shape, but no output schema exists and the complexity is low.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, and the run_id parameter is already documented as 'Run ID from kalei_profile_model.' The description repeats the resource concept but adds no parameter detail beyond what the schema provides, so the baseline of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a clear verb ('Get') and a specific resource ('final cognitive profile for a completed run'), so an agent can understand what the tool returns. It differentiates from the sibling run_progress through the word 'final,' though it does not explicitly name an alternative.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The phrase 'for a completed run' implies when this tool should be used, but the description gives no explicit guidance about alternatives such as kalei_run_progress or when not to use it. The usage context is implied rather than stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
kalei_run_startA
Start profiling YOURSELF. Returns a runId. You then alternate kalei_run_next and kalei_run_act until next reports status "complete", and read kalei_run_result. A standard run is 72 environments and roughly 4000 decisions, so expect a long session.
| Name | Required | Description | Default |
|---|---|---|---|
| depth | No | standard 72 environments, deep 82, full 164. deep needs Pro tier, full needs Enterprise. | |
| agent_id | Yes | Stable id for you. Runs sharing an agent_id aggregate, and an agent is only ranked from its second run onward. | |
| agent_name | No | Display name | |
| dimensions | No | Optional. Restrict the run to these dimensions and play ALL of their environments. ["conflict"] is 6 environments and is the cheap way to try this. | |
| agent_model | No | The model you are running on, e.g. deepseek-v4-flash |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses that the tool returns a runId, involves a long multi-step session, and typically runs 72 environments with roughly 4000 decisions. It does not mention side effects, rate limits, or cost, but the provided behavioral context is substantive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three dense sentences with no filler. The core action and return value are front-loaded, followed by workflow and expectations. Every sentence contributes needed information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The workflow, duration, and end condition are specified, and the schema covers parameter details. It is slightly incomplete because it does not mention the progress tool or clarify behavior if the run is long-running in a background sense, but it is adequate for an agent to call the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all parameters. The description adds no additional parameter-level meaning beyond what the schema provides, which matches the baseline.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Start profiling YOURSELF') and the resource (a profiling run), and indicates the return value (runId). It references sibling tools in the workflow, though it does not explicitly distinguish itself from kalei_profile_model.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit workflow usage: start, alternate next/act until complete, then read result. It gives expected scale and session length, but does not mention alternatives or exclusions such as when to prefer kalei_profile_model.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
13 tool updates
v0.2.0- First observed
kalei_ask_parliament - First observed
kalei_compare - First observed
kalei_dimensions - First observed
kalei_insights - First observed
kalei_leaderboard - First observed
kalei_model - First observed
kalei_profile_model - First observed
kalei_providers - First observed
kalei_run_act - First observed
kalei_run_next - First observed
kalei_run_progress - First observed
kalei_run_result - First observed
kalei_run_start
TDQS
Scored across 13 tools
Most tools are distinct, but kalei_profile_model and kalei_run_start both start profiling runs and return a run ID, while kalei_run_result and kalei_model both return cognitive profile data. The interactive self-run workflow (run_start/run_next/run_act) is clearly explained, but an agent could still misroute calls between the two run modes.
All tools share the kalei_ prefix, which gives some coherence, but the naming style is mixed: some are verb-oriented (run_start, run_next, run_act, compare), while others are noun-oriented (leaderboard, model, dimensions, providers). The inconsistent verb_noun pattern makes the API less predictable than it could be.
At 13 tools, the set is within a reasonable range and each tool supports a coherent profiling/leaderboard domain. The count is slightly heavier than necessary due to two separate run workflows, but no tool feels purely redundant.
The tool surface covers the core lifecycle: starting runs, monitoring progress, retrieving results, viewing leaderboards, comparing models, and listing dimensions/providers. Minor gaps exist, such as no way to cancel or list past runs, but agents can accomplish the main tasks without dead ends.
Maintenance
Related MCP Connectors
- UnifAPIOAuthcom.unifapi
Hosted MCP server for live public-data APIs and Skills for AI agents.
Use AI models for chat, image, and video generation from Claude Code and other MCP hosts.
AgencyAI's public MCP for service discovery and AI-readiness assessment.
Read-only MCP for AI usage profiles, leaderboards, stats, and docs; no writes or private data.
Related MCP Servers
- AlicenseAqualityDmaintenanceProvides tools for evaluating and benchmarking AI explanation methods through a standard interface that can be used with AI assistants and MCP-compatible applications.11MIT
- AlicenseBqualityCmaintenanceProvides AI assistants with access to Civitai's collection of AI models, enabling users to browse, search, and discover AI models through MCP-compatible AI assistants.21412MIT
- AlicenseAqualityCmaintenanceEnables AI agents to programmatically inspect, test, and validate other MCP servers by exposing MCP Workbench capabilities as structured tools. It supports automated test spec generation, execution, and detailed failure analysis to ensure server reliability.45 npmApache 2.0
- FlicenseDqualityBmaintenanceExposes the Toggle Brain knowledge base, prompt templates, and executable scripts to any MCP-compatible AI assistant.191-