Skip to main content
Glama
deenesjakoruzh
disler

Just Prompt

by disler
OverviewSchemaSourceRelated ServersDiscussionsAdmin
Python
Remote

Just Prompt - A lightweight MCP server for LLM providers

just-prompt is a Model Control Protocol (MCP) server that provides a unified interface to various Large Language Model (LLM) providers including OpenAI, Anthropic, Google Gemini, Groq, DeepSeek, and Ollama. See how we use the ceo_and_board tool to make hard decisions easy with o3 here.

Tools

The following MCP tools are available in the server:

  • prompt: Send a prompt to multiple LLM models

    • Parameters:

      • text: The prompt text

      • models_prefixed_by_provider (optional): List of models with provider prefixes. If not provided, uses default models.

  • prompt_from_file: Send a prompt from a file to multiple LLM models

    • Parameters:

      • abs_file_path: Absolute path to the file containing the prompt (must be an absolute path, not relative)

      • models_prefixed_by_provider (optional): List of models with provider prefixes. If not provided, uses default models.

  • prompt_from_file_to_file: Send a prompt from a file to multiple LLM models and save responses as markdown files

    • Parameters:

      • abs_file_path: Absolute path to the file containing the prompt (must be an absolute path, not relative)

      • models_prefixed_by_provider (optional): List of models with provider prefixes. If not provided, uses default models.

      • abs_output_dir (default: "."): Absolute directory path to save the response markdown files to (must be an absolute path, not relative)

  • ceo_and_board: Send a prompt to multiple 'board member' models and have a 'CEO' model make a decision based on their responses

    • Parameters:

      • abs_file_path: Absolute path to the file containing the prompt (must be an absolute path, not relative)

      • models_prefixed_by_provider (optional): List of models with provider prefixes to act as board members. If not provided, uses default models.

      • abs_output_dir (default: "."): Absolute directory path to save the response files and CEO decision (must be an absolute path, not relative)

      • ceo_model (default: "openai:o3"): Model to use for the CEO decision in format "provider:model"

  • list_providers: List all available LLM providers

    • Parameters: None

  • list_models: List all available models for a specific LLM provider

    • Parameters:

      • provider: Provider to list models for (e.g., 'openai' or 'o')

Related MCP server: gemini-bridge

Provider Prefixes

every model must be prefixed with the provider name

use the short name for faster referencing

  • o or openai: OpenAI

    • o:gpt-4o-mini

    • openai:gpt-4o-mini

  • a or anthropic: Anthropic

    • a:claude-3-5-haiku

    • anthropic:claude-3-5-haiku

  • g or gemini: Google Gemini

    • g:gemini-2.5-pro-exp-03-25

    • gemini:gemini-2.5-pro-exp-03-25

  • q or groq: Groq

    • q:llama-3.1-70b-versatile

    • groq:llama-3.1-70b-versatile

  • d or deepseek: DeepSeek

    • d:deepseek-coder

    • deepseek:deepseek-coder

  • l or ollama: Ollama

    • l:llama3.1

    • ollama:llama3.1

Features

  • Unified API for multiple LLM providers

  • Support for text prompts from strings or files

  • Run multiple models in parallel

  • Automatic model name correction using the first model in the --default-models list

  • Ability to save responses to files

  • Easy listing of available providers and models

Installation

# Clone the repository
git clone https://github.com/yourusername/just-prompt.git
cd just-prompt

# Install with pip
uv sync

Environment Variables

Create a .env file with your API keys (you can copy the .env.sample file):

cp .env.sample .env

Then edit the .env file to add your API keys (or export them in your shell):

OPENAI_API_KEY=your_openai_api_key_here
ANTHROPIC_API_KEY=your_anthropic_api_key_here
GEMINI_API_KEY=your_gemini_api_key_here
GROQ_API_KEY=your_groq_api_key_here
DEEPSEEK_API_KEY=your_deepseek_api_key_here
OLLAMA_HOST=http://localhost:11434

Claude Code Installation

In all these examples, replace the directory with the path to the just-prompt directory.

Default models set to openai:o3:high, openai:o4-mini:high, anthropic:claude-opus-4-20250514, anthropic:claude-sonnet-4-20250514, gemini:gemini-2.5-pro-preview-03-25, and gemini:gemini-2.5-flash-preview-04-17.

If you use Claude Code right out of the repository you can see in the .mcp.json file we set the default models to...

{
  "mcpServers": {
    "just-prompt": {
      "type": "stdio",
      "command": "uv",
      "args": [
        "--directory",
        ".",
        "run",
        "just-prompt",
        "--default-models",
        "openai:o3:high,openai:o4-mini:high,anthropic:claude-opus-4-20250514,anthropic:claude-sonnet-4-20250514,gemini:gemini-2.5-pro-preview-03-25,gemini:gemini-2.5-flash-preview-04-17"
      ],
      "env": {}
    }
  }
}

The --default-models parameter sets the models to use when none are explicitly provided to the API endpoints. The first model in the list is also used for model name correction when needed. This can be a list of models separated by commas.

When starting the server, it will automatically check which API keys are available in your environment and inform you which providers you can use. If a key is missing, the provider will be listed as unavailable, but the server will still start and can be used with the providers that are available.

Using mcp add-json

Copy this and paste it into claude code with BUT don't run until you copy the json

claude mcp add just-prompt "$(pbpaste)"

JSON to copy

{
    "command": "uv",
    "args": ["--directory", ".", "run", "just-prompt"]
}

With a custom default model set to openai:gpt-4o.

{
    "command": "uv",
    "args": ["--directory", ".", "run", "just-prompt", "--default-models", "openai:gpt-4o"]
}

With multiple default models:

{
    "command": "uv",
    "args": ["--directory", ".", "run", "just-prompt", "--default-models", "openai:o3:high,openai:o4-mini:high,anthropic:claude-opus-4-20250514,anthropic:claude-sonnet-4-20250514,gemini:gemini-2.5-pro-preview-03-25,gemini:gemini-2.5-flash-preview-04-17"]
}

Using mcp add with project scope

# With default models
claude mcp add just-prompt -s project \
  -- \
    uv --directory . \
    run just-prompt

# With custom default model
claude mcp add just-prompt -s project \
  -- \
  uv --directory . \
  run just-prompt --default-models "openai:gpt-4o"

# With multiple default models
claude mcp add just-prompt -s user \
  -- \
  uv --directory . \
  run just-prompt --default-models "openai:o3:high,openai:o4-mini:high,anthropic:claude-opus-4-20250514,anthropic:claude-sonnet-4-20250514,gemini:gemini-2.5-pro-preview-03-25,gemini:gemini-2.5-flash-preview-04-17"

mcp remove

claude mcp remove just-prompt

Running Tests

uv run pytest

Codebase Structure

.
├── ai_docs/                   # Documentation for AI model details
│   ├── extending_thinking_sonny.md
│   ├── llm_providers_details.xml
│   ├── openai-reasoning-effort.md
│   └── pocket-pick-mcp-server-example.xml
├── example_outputs/           # Example outputs from different models
├── list_models.py             # Script to list available LLM models
├── prompts/                   # Example prompt files
├── pyproject.toml             # Python project configuration
├── specs/                     # Project specifications
│   ├── init-just-prompt.md
│   ├── new-tool-llm-as-a-ceo.md
│   └── oai-reasoning-levels.md
├── src/                       # Source code directory
│   └── just_prompt/
│       ├── __init__.py
│       ├── __main__.py
│       ├── atoms/             # Core components
│       │   ├── llm_providers/ # Individual provider implementations
│       │   │   ├── anthropic.py
│       │   │   ├── deepseek.py
│       │   │   ├── gemini.py
│       │   │   ├── groq.py
│       │   │   ├── ollama.py
│       │   │   └── openai.py
│       │   └── shared/        # Shared utilities and data types
│       │       ├── data_types.py
│       │       ├── model_router.py
│       │       ├── utils.py
│       │       └── validator.py
│       ├── molecules/         # Higher-level functionality
│       │   ├── ceo_and_board_prompt.py
│       │   ├── list_models.py
│       │   ├── list_providers.py
│       │   ├── prompt.py
│       │   ├── prompt_from_file.py
│       │   └── prompt_from_file_to_file.py
│       ├── server.py          # MCP server implementation
│       └── tests/             # Test directory
│           ├── atoms/         # Tests for atoms
│           │   ├── llm_providers/
│           │   └── shared/
│           └── molecules/     # Tests for molecules
│               ├── test_ceo_and_board_prompt.py
│               ├── test_list_models.py
│               ├── test_list_providers.py
│               ├── test_prompt.py
│               ├── test_prompt_from_file.py
│               └── test_prompt_from_file_to_file.py
└── ultra_diff_review/         # Diff review outputs

Context Priming

READ README.md, pyproject.toml, then run git ls-files, and 'eza --git-ignore --tree' to understand the context of the project.

Reasoning Effort with OpenAI o‑Series

For OpenAI o‑series reasoning models (o4-mini, o3-mini, o3) you can control how much internal reasoning the model performs before producing a visible answer.

Append one of the following suffixes to the model name (after the provider prefix):

  • :low   – minimal internal reasoning (faster, cheaper)

  • :medium – balanced (default if omitted)

  • :high  – thorough reasoning (slower, more tokens)

Examples:

  • openai:o4-mini:low

  • o:o4-mini:high

When a reasoning suffix is present, just‑prompt automatically switches to the OpenAI Responses API (when available) and sets the corresponding reasoning.effort parameter. If the installed OpenAI SDK is older, it gracefully falls back to the Chat Completions endpoint and embeds an internal system instruction to approximate the requested effort level.

Thinking Tokens with Claude

The Anthropic Claude models claude-opus-4-20250514 and claude-sonnet-4-20250514 support extended thinking capabilities using thinking tokens. This allows Claude to do more thorough thought processes before answering.

You can enable thinking tokens by adding a suffix to the model name in this format:

  • anthropic:claude-opus-4-20250514:1k - Use 1024 thinking tokens for Opus 4

  • anthropic:claude-sonnet-4-20250514:4k - Use 4096 thinking tokens for Sonnet 4

  • anthropic:claude-opus-4-20250514:8000 - Use 8000 thinking tokens for Opus 4

Notes:

  • Thinking tokens are supported for claude-opus-4-20250514, claude-sonnet-4-20250514, and claude-3-7-sonnet-20250219 models

  • Valid thinking token budgets range from 1024 to 16000

  • Values outside this range will be automatically adjusted to be within range

  • You can specify the budget with k notation (1k, 4k, etc.) or with exact numbers (1024, 4096, etc.)

Thinking Budget with Gemini

The Google Gemini model gemini-2.5-flash-preview-04-17 supports extended thinking capabilities using thinking budget. This allows Gemini to perform more thorough reasoning before providing a response.

You can enable thinking budget by adding a suffix to the model name in this format:

  • gemini:gemini-2.5-flash-preview-04-17:1k - Use 1024 thinking budget

  • gemini:gemini-2.5-flash-preview-04-17:4k - Use 4096 thinking budget

  • gemini:gemini-2.5-flash-preview-04-17:8000 - Use 8000 thinking budget

Notes:

  • Thinking budget is only supported for the gemini-2.5-flash-preview-04-17 model

  • Valid thinking budget range from 0 to 24576

  • Values outside this range will be automatically adjusted to be within range

  • You can specify the budget with k notation (1k, 4k, etc.) or with exact numbers (1024, 4096, etc.)

Resources

Master AI Coding

Learn to code with AI with foundational Principles of AI Coding

Follow the IndyDevDan youtube channel for more AI coding tips and tricks.

Available Tools

6 tools
ceo_and_boardB

Send a prompt to multiple 'board member' models and have a 'CEO' model make a decision based on their responses. IMPORTANT: You MUST provide absolute paths (e.g., /path/to/file or C:\path\to\file) for both file and output directory, not relative paths.

ParametersJSON Schema
NameRequiredDescriptionDefault
abs_file_pathYesAbsolute path to the file containing the prompt (must be an absolute path, not relative)
abs_output_dirNoAbsolute directory path to save the response files and CEO decision (must be an absolute path, not relative).
ceo_modelNoModel to use for the CEO decision in format 'provider:model'openai:o3
models_prefixed_by_providerNoList of models with provider prefixes to act as board members. If not provided, uses default models.

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It discloses the need for absolute paths and mentions the CEO decision process, but lacks details on behavioral traits such as error handling, rate limits, authentication needs, or what the output looks like (e.g., file formats, decision format). For a tool with 4 parameters and no annotations, this is a significant gap in transparency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is appropriately sized with two sentences: the first explains the core functionality, and the second provides a critical usage note. It's front-loaded with the main purpose, and the 'IMPORTANT' section adds necessary guidance without redundancy. However, the second sentence could be integrated more smoothly, slightly affecting structure.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (multi-model decision-making with file I/O), no annotations, and no output schema, the description is incomplete. It doesn't explain the output format, how the CEO decision is derived, error cases, or dependencies on other tools. For a tool with this functionality, more context is needed to ensure proper usage by an AI agent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds minimal value beyond the schema by emphasizing absolute paths in a note, but doesn't provide additional semantic context like examples or rationale for parameter choices. With high schema coverage, the baseline is 3, and the description meets this without compensating further.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Send a prompt to multiple 'board member' models and have a 'CEO' model make a decision based on their responses.' It specifies the verb ('send'), resource ('prompt'), and outcome ('CEO model make a decision'), distinguishing it from simpler prompt tools like 'prompt' or 'prompt_from_file'. However, it doesn't explicitly differentiate from 'prompt_from_file_to_file' which also involves file-based prompting with output.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage when needing multi-model consensus with a CEO decision-maker, as opposed to single-model prompts. It includes an 'IMPORTANT' note about absolute paths, which provides some context. However, it doesn't explicitly state when to use this tool versus alternatives like 'prompt_from_file_to_file' or under what scenarios the board/CEO metaphor is beneficial, leaving usage somewhat implied rather than clearly defined.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_modelsC

List all available models for a specific LLM provider

ParametersJSON Schema
NameRequiredDescriptionDefault
providerYesProvider to list models for (e.g., 'openai' or 'o')

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It states it's a list operation (implied read-only) but doesn't disclose important behavioral traits like whether it requires authentication, rate limits, pagination behavior, error handling, or what format the returned models list will have. The description is minimal and lacks operational context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. However, it could be more front-loaded with critical information about behavioral aspects given the lack of annotations.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with no annotations, no output schema, and minimal description, the contextual information is insufficient. The description doesn't explain what 'available models' means (e.g., supported models, all models including deprecated ones), doesn't describe the return format, and provides no error handling or authentication context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema description coverage is 100%, so the input schema already fully documents the single 'provider' parameter with examples. The description adds no additional parameter semantics beyond what's in the schema, maintaining the baseline score of 3 for adequate coverage when schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('List all available models') and the target resource ('for a specific LLM provider'), providing a specific verb+resource combination. However, it doesn't differentiate from sibling tools like 'list_providers' which might be conceptually related but serves a different function.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives, prerequisites, or contextual constraints. It mentions 'specific LLM provider' but doesn't explain how to determine which provider to use or what happens if an invalid provider is specified.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_providersB

List all available LLM providers

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries full burden for behavioral disclosure. It states what the tool does but provides no information about permissions needed, rate limits, pagination behavior, response format, or whether this is a read-only operation. For a tool with zero annotation coverage, this leaves significant behavioral gaps unaddressed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that communicates the core purpose without any wasted words. It's appropriately sized for a simple list operation and front-loads the essential information. Every word earns its place in this minimal description.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool has no annotations, no output schema, and the description provides only basic purpose information, there are significant completeness gaps. For even a simple list operation, the description should address response format, potential limitations, or behavioral context. The current description is insufficient for a tool that agents need to understand fully before invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters with 100% schema description coverage, so the schema already fully documents the parameter situation. The description appropriately doesn't discuss parameters since none exist. This earns a baseline score of 4 for parameter semantics when there are no parameters to document.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb ('List') and resource ('all available LLM providers'), making the purpose immediately understandable. It doesn't explicitly differentiate from sibling tools like 'list_models', but the resource specificity (providers vs models) provides implicit differentiation. The description avoids tautology by not just restating the tool name.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives like 'list_models' or other sibling tools. It doesn't mention prerequisites, context for usage, or any exclusions. While the purpose is clear, there's no explicit usage guidance beyond the basic action described.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

promptC

Send a prompt to multiple LLM models

ParametersJSON Schema
NameRequiredDescriptionDefault
models_prefixed_by_providerNoList of models with provider prefixes (e.g., 'openai:gpt-4o' or 'o:gpt-4o'). If not provided, uses default models.
textYesThe prompt text

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It states the action ('send a prompt') but lacks details on what happens: e.g., how models are selected, whether responses are returned or stored, any rate limits, authentication needs, or error handling. This is a significant gap for a tool interacting with external LLMs.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence with zero waste. It's front-loaded and directly states the tool's function without unnecessary elaboration, making it highly concise and well-structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity of interacting with multiple LLM models, no annotations, and no output schema, the description is incomplete. It doesn't cover behavioral aspects like response format, error handling, or model selection logic, leaving gaps that could hinder effective tool use by an AI agent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema fully documents both parameters. The description adds no additional meaning beyond what's in the schema (e.g., it doesn't explain the 'models_prefixed_by_provider' format further or provide examples beyond the schema's description). Baseline 3 is appropriate as the schema handles parameter documentation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Send a prompt to multiple LLM models' clearly states the action (send) and resource (prompt to LLM models), making the purpose understandable. However, it doesn't differentiate from sibling tools like 'prompt_from_file' or 'prompt_from_file_to_file', which also involve sending prompts but with different input methods.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. It doesn't mention when to choose this over 'prompt_from_file' (for file-based prompts) or 'prompt_from_file_to_file' (for file-to-file processing), nor does it specify any prerequisites or exclusions for usage.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

prompt_from_fileA

Send a prompt from a file to multiple LLM models. IMPORTANT: You MUST provide an absolute file path (e.g., /path/to/file or C:\path\to\file), not a relative path.

ParametersJSON Schema
NameRequiredDescriptionDefault
abs_file_pathYesAbsolute path to the file containing the prompt (must be an absolute path, not relative)
models_prefixed_by_providerNoList of models with provider prefixes (e.g., 'openai:gpt-4o' or 'o:gpt-4o'). If not provided, uses default models.

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It adds value by specifying the absolute path requirement and hinting at default model behavior if 'models_prefixed_by_provider' is not provided. However, it lacks details on error handling, rate limits, authentication needs, or output format, leaving gaps in behavioral transparency for a tool that interacts with LLMs.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is appropriately sized and front-loaded, with two sentences that directly convey the tool's purpose and a critical requirement. Every sentence earns its place by providing essential information without redundancy, making it efficient and easy to parse.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's moderate complexity (interacting with LLMs), no annotations, and no output schema, the description is incomplete. It covers the basic operation and path requirement but lacks details on output format, error handling, or model behavior, which are important for effective use. This is adequate as a minimum viable description but has clear gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents both parameters thoroughly. The description adds minimal semantic context beyond the schema, such as emphasizing the absolute path requirement and giving examples for model prefixes. This meets the baseline of 3, as the schema does the heavy lifting, but doesn't provide significant additional meaning.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Send a prompt from a file to multiple LLM models.' It specifies the verb ('send'), resource ('prompt from a file'), and target ('multiple LLM models'), making it easy to understand what the tool does. However, it doesn't explicitly differentiate from sibling tools like 'prompt' or 'prompt_from_file_to_file', which would require a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides some usage guidance by emphasizing the requirement for an absolute file path, but it doesn't explicitly state when to use this tool versus alternatives like 'prompt' (which might accept direct text input) or 'prompt_from_file_to_file' (which might output to a file). The guidance is implied rather than explicit, falling short of the highest scores.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

prompt_from_file_to_fileB

Send a prompt from a file to multiple LLM models and save responses to files. IMPORTANT: You MUST provide absolute paths (e.g., /path/to/file or C:\path\to\file) for both file and output directory, not relative paths.

ParametersJSON Schema
NameRequiredDescriptionDefault
abs_file_pathYesAbsolute path to the file containing the prompt (must be an absolute path, not relative)
abs_output_dirNoAbsolute directory path to save the response files to (must be an absolute path, not relative. Default: current directory).
models_prefixed_by_providerNoList of models with provider prefixes (e.g., 'openai:gpt-4o' or 'o:gpt-4o'). If not provided, uses default models.

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries full burden for behavioral disclosure. It mentions the absolute path requirement (a constraint) and that it processes 'multiple LLM models', but doesn't describe what happens during execution (e.g., sequential/parallel processing, error handling, file naming conventions, or what 'default models' means). For a tool with file I/O and model execution, this leaves significant behavioral gaps unaddressed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is appropriately sized (two sentences) and front-loaded with the core purpose. The 'IMPORTANT' note is relevant but could be integrated more smoothly. There's no wasted text, and every sentence adds value (purpose and critical constraint), though the structure is slightly abrupt with the all-caps emphasis.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (file I/O, model execution, batch processing) with no annotations and no output schema, the description is incomplete. It doesn't explain what the output files contain (e.g., raw responses, metadata), how errors are handled, what 'default models' are, or the execution behavior. For a tool with multiple parameters and significant side effects, this leaves too much unspecified for reliable agent use.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds minimal value beyond the schema: it reinforces the absolute path requirement (already in schema descriptions) and mentions 'multiple LLM models' (implied by the array parameter). No additional syntax, format, or semantic details are provided beyond what's in the schema descriptions, meeting the baseline for high coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Send a prompt from a file to multiple LLM models and save responses to files') with specific resources (file input, file output, LLM models). It distinguishes from sibling 'prompt' (which likely takes direct input) and 'prompt_from_file' (which likely doesn't save to files), but doesn't explicitly name these alternatives. The purpose is specific but could be more precise about sibling differentiation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage context through the 'IMPORTANT' note about absolute paths, suggesting this tool is for file-based batch processing. However, it doesn't explicitly state when to use this vs. 'prompt_from_file' (which likely processes from file but doesn't save to files) or 'prompt' (direct input). No explicit alternatives or exclusions are provided, leaving usage context somewhat implied rather than clearly defined.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 6 tool updatesv1.0.0
    • First observedceo_and_board
    • First observedlist_models
    • First observedlist_providers
    • First observedprompt
    • First observedprompt_from_file
    • First observedprompt_from_file_to_file
Deploy Server
One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

TDQS

B3.3/5.0

Scored across 6 tools

Disambiguation3/5

There is significant overlap between 'prompt', 'prompt_from_file', and 'prompt_from_file_to_file', all centered on sending prompts to multiple models, which could cause confusion. However, 'ceo_and_board' adds a distinct decision-making layer, and 'list_models' and 'list_providers' are clearly separate informational tools, helping to mitigate ambiguity.

Naming Consistency4/5

Most tools follow a clear verb_noun or verb_from_noun pattern (e.g., 'list_models', 'prompt_from_file'), with consistent snake_case throughout. The only deviation is 'ceo_and_board', which uses a noun-based name that breaks the verb-led convention, but it's still readable and not chaotic.

Tool Count5/5

With 6 tools, this server is well-scoped for its purpose of managing and prompting LLM models. The count is sufficient to cover core functionalities like listing providers/models and various prompting methods without being overwhelming or too sparse.

Completeness4/5

The toolset covers key operations for LLM interaction: listing providers and models, basic prompting, file-based prompting, and an advanced decision-making tool. A minor gap exists in lacking update or delete operations for prompts or models, but agents can likely work around this for typical use cases.

Maintenance

ActivityInactive
ResponsivenessUnresponsive

Resources

Related MCP Connectors

Related MCP Servers

Appeared in Searches