Skip to main content
Glama
mcp-units

mcp-units

by quantumleeps

mcp-units

An MCP server that provides deterministic unit conversions via Pint. LLMs guess at unit conversions — this server makes them exact.

What this does

Exposes 5 tools, 3 resources, and 2 prompts over the Model Context Protocol. Any MCP client (Claude Code, Claude Desktop, Cursor) can convert units, check dimensional compatibility, parse quantity strings, and simplify expressions — all backed by Pint's 400+ unit registry instead of LLM arithmetic.

Related MCP server: MCP Mathematics

How it works

A FastMCP server wraps Pint's UnitRegistry and exposes it through MCP primitives:

  • Tools — convert, check_compatibility, parse_quantity, list_compatible_units, simplify

  • Resources — units://systems, units://systems/{system}, units://dimensions

  • Prompts — convert_document (extract and convert all quantities in text), check_calculations (verify dimensional consistency)

The server runs over stdio by default (for Claude Code / Claude Desktop) or Streamable HTTP via fastmcp run (for remote / containerized deployment).

Quickstart

Prerequisites

  • Python 3.12+

  • uv

Install and run

git clone https://github.com/quantumleeps/mcp-units.git
cd mcp-units
uv sync

Add to Claude Code

claude mcp add --transport stdio mcp-units -- \
  uv run --directory /path/to/mcp-units mcp-units

Add to Claude Desktop

Add to claude_desktop_config.json:

{
  "mcpServers": {
    "mcp-units": {
      "command": "uv",
      "args": ["run", "--directory", "/path/to/mcp-units", "mcp-units"]
    }
  }
}

Run over HTTP

uv run fastmcp run src/mcp_units/server.py --transport http --port 8000

Docker

docker build -t mcp-units .
docker run -p 8000:8000 mcp-units

Tests

uv sync --all-extras
uv run pytest

Evaluation

Does giving an LLM access to a unit conversion tool actually improve its accuracy on physics problems?

Tool impact across 6 Claude models

Evaluated on 70 SciBench college-level physics problems requiring 2+ unit types, across 6 Claude models (840 total runs). Opus 4.6 — the latest model — shows the largest gain (+8.6pp, 70.0% → 78.6%), suggesting that its combination of broad knowledge and refined tool-use lets it leverage unit conversion as a reliable augmentation. 4.5-Sonnet, a strong reasoner and tool user, also improves (+2.9pp). The older 3.7-Sonnet regresses (-2.9pp) — analysis shows it sometimes treats an intermediate conversion result as the final answer, or spins through repeated tool calls without converging, consistent with less mature tool-use capabilities. The surprise is 4.5-Haiku: same generation as 4.5-Sonnet with capable reasoning and tool use, yet it declines (-1.4pp). With a smaller model, the tool appears to be a distraction rather than an augmentation — the model has the sophistication to use it but not always the judgment to know when it helps. With only 70 problems and a single run per model, these per-model deltas carry real uncertainty — the 4.5-Haiku result in particular could reflect noise rather than a meaningful pattern.

Next steps

  • Unit normalization — Models write cm3 but Pint needs cm^3. A lightweight normalize_unit() preprocessor plus better tool descriptions with formatting guidance would eliminate the 12 parsing failures observed in the eval.

  • Expression evaluation — Models sometimes pass math expressions (-1.602e-19 * 1.33e-39 / ...) as the value parameter to convert(). Pint rejects these since it expects a float. Accepting and evaluating simple arithmetic expressions would let the tool handle intermediate calculations.

  • Offset unit handling — Pint raises OffsetUnitCalculusError for °C and °F in compound expressions. The parse_quantity tool needs special handling for temperature offsets.

  • Larger problem set — 70 problems demonstrates the evaluation framework but limits statistical confidence on per-model deltas. Run-to-run variance within a single model is also unknown. Expanding to 200+ problems with multiple runs per problem would quantify both effects.

Run the eval

uv sync --group eval
uv run python -m eval.runner          # run all 6 models × 2 conditions (requires ANTHROPIC_API_KEY)
uv run python -m eval.visualize       # generate charts from results
uv run python -m eval.analyze         # print detailed analysis

Project Structure

mcp-units/
  src/mcp_units/
    server.py       # FastMCP instance — tools, resources, prompts
    registry.py     # Pint UnitRegistry + compatible units workaround
    models.py       # Result dataclasses for structured tool output
  eval/
    runner.py       # Async eval runner — baseline vs tool-augmented
    problems.py     # SciBench problem loading (70 problems, 2+ unit types)
    scorer.py       # Answer extraction + 5% tolerance scoring
    mcp_tools.py    # FastMCP Client wrapper for tool execution
    results.py      # RunResult dataclass + JSON persistence
    visualize.py    # Grouped bar chart + error histograms
    analyze.py      # 16-section detailed analysis
  tests/
    test_tools.py   # 18 Pint logic tests
    test_server.py  # 17 MCP Client integration tests
  Dockerfile        # HTTP transport for containerized deployment

Contributing

PRs welcome. Run pre-commit install after cloning and ensure uv run pytest passes before submitting.

License

MIT

Available Tools

5 tools
check_compatibilityB

Check if two units are dimensionally compatible (i.e., can be converted).

Returns whether the units share the same physical dimension.

Inputs: unit_a (required), unit_b (required). Returns: unit_a, unit_b, compatible, explanation, +2 more

convertA

Convert a value from one unit to another.

Returns the converted value with conversion factor, or a structured error if the units are dimensionally incompatible.

Inputs: value (required), to_unit (required), from_unit (required). Returns: result

list_compatible_unitsA

List all units compatible with the given unit.

Returns every canonical unit name that shares the same physical dimension, including imperial and US customary units.

Inputs: unit (required). Returns: result

parse_quantityA

Parse a quantity string into structured components.

Accepts expressions like '100 mg/L' or '9.81 m/s²'. Returns the magnitude, units, dimensionality, and SI equivalent.

Inputs: expression (required). Returns: result

simplifyA

Simplify a unit expression to its most compact form.

Adjusts prefixes (e.g., 1000 Pa → 1 kPa) and reduces named units. Also provides the base SI representation.

Inputs: expression (required). Returns: result

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    A3.9/5.0

    Scored across 5 tools

    Disambiguation5/5

    Each tool has a distinct, clearly defined purpose: checking compatibility, converting, listing compatible units, parsing strings, and simplifying expressions. No overlapping functionality.

    Naming Consistency4/5

    All tools use snake_case and follow a verb-based naming pattern. Most are verb_noun (check_compatibility, list_compatible_units, parse_quantity), while convert and simplify are single verbs, which is a minor inconsistency.

    Tool Count5/5

    With 5 tools, the server is well-scoped for unit operations. Each tool addresses a core need without unnecessary bloat.

    Completeness4/5

    The set covers essential operations: compatibility check, conversion, listing compatible units, parsing, and simplification. A minor gap is the lack of a tool to retrieve detailed unit metadata, but this is not critical.

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      A
      quality
      C
      maintenance
      A small MCP server for the boring-but-essential utilities every model needs: dates, calendars, arithmetic, unit conversion. Use it so your assistant stops "next-token guessing" math and date math.
      4
      39 PyPI
      1
      MIT
    • A
      license
      C
      quality
      D
      maintenance
      A comprehensive MCP server that turns any AI assistant into a powerful mathematical computation engine, providing 52 advanced functions, 158 unit conversions, financial calculations, and secure AST-based evaluation.
      18
      16
      MIT
    • A
      license
      Not graded
      quality
      D
      maintenance
      An MCP server for chemistry-focused tools, enabling LLM agents to perform molecule parsing, format conversion, property lookup, and other chemistry operations with explainable responses.
      Apache 2.0