Skip to main content
Glama
guidance-ai

MCP Grammar Tools

by guidance-ai
OverviewSchemaSourceRelated ServersDiscussionsAdmin
Python
Local

MCP Grammar Tools

MCP server for validating and testing llguidance grammars (Lark format). Provides grammar validation, batch test execution, and syntax documentation — ideal for iteratively building grammars with AI coding assistants.

Installation

With uvx (recommended)

uvx guidance-lark-mcp

With pip

pip install guidance-lark-mcp

From source

cd mcp-grammar-tools
pip install -e .

Related MCP server: Fast Mermaid Validator MCP

MCP Client Configuration

GitHub Copilot CLI

You can add the server using the interactive /mcp add command or by editing the config file directly. See the Copilot CLI MCP documentation for full details.

Option 1: Interactive setup

In the Copilot CLI, run /mcp add, select Local/STDIO, and enter uvx guidance-lark-mcp as the command.

Option 2: Edit config file

Add the following to ~/.copilot/mcp-config.json:

{
  "mcpServers": {
    "grammar-tools": {
      "type": "local",
      "command": "uvx",
      "args": ["guidance-lark-mcp"],
      "tools": ["*"]
    }
  }
}

This gives you grammar validation and batch testing out of the box. To also enable LLM-powered generation (generate_with_grammar), add ENABLE_GENERATION and your credentials to env:

"env": {
  "ENABLE_GENERATION": "true",
  "OPENAI_API_KEY": "your-key-here"
}

For Azure OpenAI (with Entra ID via az login), use guidance-lark-mcp[azure] and set the endpoint instead:

"args": ["guidance-lark-mcp[azure]"],
"env": {
  "ENABLE_GENERATION": "true",
  "AZURE_OPENAI_ENDPOINT": "https://your-resource.openai.azure.com/",
  "OPENAI_MODEL": "your-deployment-name"
}

See Backend Configuration for all supported backends.

After saving, use /mcp show to verify the server is connected.

VS Code

{
  "mcpServers": {
    "grammar-tools": {
      "type": "local",
      "command": "uvx",
      "args": ["guidance-lark-mcp"],
      "env": {
        "ENABLE_GENERATION": "true",
        "OPENAI_API_KEY": "your-key-here"
      },
      "tools": ["*"]
    }
  }
}

Claude Desktop

{
  "mcpServers": {
    "grammar-tools": {
      "command": "uvx",
      "args": ["guidance-lark-mcp"],
      "env": {
        "ENABLE_GENERATION": "true",
        "OPENAI_API_KEY": "your-key-here"
      }
    }
  }
}

Usage

Available Tools

  1. validate_grammar — Validate grammar completeness and consistency using llguidance's built-in validator.

    {"grammar": "start: \"hello\" \"world\""}

  2. run_batch_validation_tests — Run batch validation tests from a JSON file against a grammar. Returns pass/fail statistics and detailed failure info.

    {
  "grammar": "start: /[0-9]+/",
  "test_file": "tests.json"
}

    Test file format:

    [
  {"input": "123", "should_parse": true, "description": "Valid number"},
  {"input": "abc", "should_parse": false, "description": "Not a number"}
]

  3. get_llguidance_documentation — Fetch the llguidance grammar syntax documentation from the official repo.

  4. generate_with_grammar (optional, requires ENABLE_GENERATION=true) — Generate text using an OpenAI model constrained by a grammar. Uses the Responses API with custom tool grammar format, so output is guaranteed to conform to the grammar. Requires OPENAI_API_KEY environment variable. See Backend Configuration for Azure and other endpoints.

Backend Configuration

The generate_with_grammar tool uses the OpenAI Python SDK, which natively supports multiple backends via environment variables:

Backend

Required env vars

Optional env vars

OpenAI (default)

OPENAI_API_KEY

OPENAI_MODEL

Azure OpenAI (API key)

AZURE_OPENAI_ENDPOINT, AZURE_OPENAI_API_KEY

AZURE_OPENAI_API_VERSION, OPENAI_MODEL

Azure OpenAI (Entra ID)

AZURE_OPENAI_ENDPOINT + az login

AZURE_OPENAI_API_VERSION, OPENAI_MODEL

Custom endpoint

OPENAI_API_KEY, OPENAI_BASE_URL

OPENAI_MODEL

The server auto-detects which backend to use:

  • If AZURE_OPENAI_ENDPOINT is set → uses AzureOpenAI client (with Entra ID or API key)

  • Otherwise → uses OpenAI client (reads OPENAI_API_KEY and OPENAI_BASE_URL automatically)

The server logs which backend it detects on startup.

Example: Azure OpenAI (API key)

{
  "mcpServers": {
    "grammar-tools": {
      "type": "local",
      "command": "uvx",
      "args": ["guidance-lark-mcp"],
      "env": {
        "ENABLE_GENERATION": "true",
        "AZURE_OPENAI_ENDPOINT": "https://my-resource.openai.azure.com",
        "AZURE_OPENAI_API_KEY": "your-azure-key",
        "OPENAI_MODEL": "gpt-4.1"
      },
      "tools": ["*"]
    }
  }
}

Example: Azure OpenAI (Entra ID / keyless)

Requires az login and the azure extra: pip install guidance-lark-mcp[azure]

{
  "mcpServers": {
    "grammar-tools": {
      "type": "local",
      "command": "uvx",
      "args": ["guidance-lark-mcp[azure]"],
      "env": {
        "ENABLE_GENERATION": "true",
        "AZURE_OPENAI_ENDPOINT": "https://my-resource.openai.azure.com",
        "OPENAI_MODEL": "gpt-4.1"
      },
      "tools": ["*"]
    }
  }
}

Example Workflow

Build a grammar iteratively with an AI assistant:

  1. Start with the spec — paste EBNF rules from a language specification

  2. Write a basic grammar — translate a few rules to Lark format

  3. Validate — use validate_grammar to check for missing rules

  4. Write tests — create a JSON test file with sample inputs

  5. Batch test — use run_batch_validation_tests to find failures

  6. Fix & repeat — refine the grammar until all tests pass

Example Grammars

The examples/ directory includes sample grammars built using these tools, with Lark grammar files, test suites, and documentation:

  • GraphQL — executable subset of the GraphQL spec (queries, mutations, fragments, variables)

Troubleshooting

Server fails to connect in Copilot CLI / VS Code?

MCP clients like Copilot CLI only show "Connection closed" when a server crashes on startup. To see the actual error, run the server directly in your terminal:

uvx guidance-lark-mcp

Or with generation enabled:

ENABLE_GENERATION=true OPENAI_API_KEY=your-key uvx guidance-lark-mcp

Common issues:

  • Missing credentials — ENABLE_GENERATION=true without a valid OPENAI_API_KEY or AZURE_OPENAI_ENDPOINT. The server will still start and serve validation tools; generate_with_grammar will return a descriptive error.

  • Azure Entra ID — make sure you've run az login and are using guidance-lark-mcp[azure] (not the base package).

  • Slow first start — uvx needs to resolve and install dependencies on first run, which may exceed the MCP client's connection timeout. Run uvx guidance-lark-mcp once manually to warm the cache.

  • Updating to a new version — uvx caches packages, so after a new release you may need to clear the cache and restart your MCP client:

    uv cache clean guidance-lark-mcp

Development

git clone https://github.com/guidance-ai/guidance-lark-mcp
cd guidance-lark-mcp
uv sync
uv run pytest tests/ -q

Available Tools

3 tools
get_llguidance_documentationA

Fetch the llguidance grammar syntax documentation.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A3.5/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations exist, so the description must convey behavior. It only states 'Fetch' which implies read-only, but no details on output format, side effects, or whether it requires any state. Minimal transparency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Single sentence, no extraneous words, clear and front-loaded. Efficient use of text.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no parameters and no output schema, the description is adequate but minimal. It lacks details on the return format or any prerequisites. A bit more context would be helpful but not critical.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

No parameters defined, so schema coverage is 100%. The description adds no parameter info, but with zero parameters, a baseline of 4 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb 'Fetch' and identifies the resource as 'llguidance grammar syntax documentation'. It clearly distinguishes from siblings like 'validate_grammar' which validates, and 'run_batch_validation_tests' which runs tests.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this tool versus alternatives. The description is minimal and does not provide context for appropriate usage or exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

run_batch_validation_testsA

Run batch validation tests from a JSON file against an llguidance grammar. Test file should contain array of {input, should_parse, description?} objects. Returns high-level statistics (total, passed, failed, success_rate) and detailed information for failed tests.

ParametersJSON Schema
NameRequiredDescriptionDefault
grammarYesllguidance grammar string or path to grammar file (.lark, .grammar)
test_fileYesPath to JSON test file with format: [{"input": "test", "should_parse": true}] or {"tests": [...]}

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Describes expected behavior (runs tests, returns statistics and fail details) but does not disclose potential side effects, error conditions, or performance implications. With no annotations, description carries full burden, yet remains basic.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with purpose, no extraneous information. Every sentence adds value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Covers purpose, parameter basics, and return format. With no output schema, the description adequately explains what is returned. Lacks minor details like path resolution or error handling, but sufficient for a simple tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema already covers both parameters with good descriptions. Description adds minimal new parameter information beyond restating test file format. Baseline 3 due to 100% schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clearly states the verb 'run', resource 'batch validation tests', and context 'from a JSON file against an llguidance grammar'. Distinguishes from siblings 'get_llguidance_documentation' and 'validate_grammar' by focusing on test execution.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Implies usage context (when you have a grammar and test file), but no explicit when-not or alternatives. Purpose is distinct enough from siblings that agent can infer correct use, lacking explicit guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

validate_grammarA

Validate llguidance grammar completeness and consistency using llguidance's built-in validator. Returns validation result with any errors or warnings.

ParametersJSON Schema
NameRequiredDescriptionDefault
grammarYesllguidance grammar string or path to grammar file (.lark, .grammar)

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so description carries full burden. It mentions validation and returning errors/warnings but does not specify side effects, permissions, or read-only nature. Adequate but lacks deeper behavioral context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences with front-loaded action. No unnecessary words or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given low complexity (single param, no output schema), the description covers the core functionality and output. Could mention output format or version dependencies, but not essential.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Only one parameter with 100% schema description coverage. The tool description does not add any meaning beyond the schema's description of 'grammar string or path'. Baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool validates llguidance grammar completeness and consistency using a built-in validator, distinguishing it from sibling tools like get_llguidance_documentation and run_batch_validation_tests.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for validating a single grammar but does not explicitly state when to use it over siblings or when not to use it. Sibling tool names provide context but no explicit guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 3 tool updatesv0.1.0
    • First observedget_llguidance_documentation
    • First observedrun_batch_validation_tests
    • First observedvalidate_grammar
Deploy Server
RayrunBuild, deploy, and operate MCP servers without becoming an infrastructure team.ray.runAd

TDQS

A4/5.0

Scored across 3 tools

Disambiguation5/5

Each tool has a distinct purpose: documentation retrieval, batch testing, and grammar validation. There is no overlap or ambiguity.

Naming Consistency5/5

All tool names follow a consistent verb_noun pattern (get_, run_, validate_) with clear and descriptive names.

Tool Count5/5

Three tools is appropriate for a focused grammar validation and testing server, covering the core functionalities without being overly sparse.

Completeness4/5

The server covers documentation retrieval, validation, and batch testing. A minor gap is the lack of grammar editing or listing tools, but the core use case is well supported.

Maintenance

ActivityInactive
ResponsivenessNo issues

Resources

Related MCP Connectors

Related MCP Servers