Skip to main content
Glama
talk-to-your-metrics

talk-to-your-metrics

by Abiramashree

talk-to-your-metrics

Ask an AI agent a business question and it writes SQL and returns a number. On real data the number is often wrong, not because the SQL fails, but because the question has more than one reasonable answer and the agent quietly picks one.

On the Olist e-commerce data (99,441 orders, 2016–2018):

Question

Answers a reasonable query can return

What was total revenue?

13.22M, 13.59M, 15.84M or 16.01M, depending on freight, payments and canceled orders

How many customers ordered more than once?

0 or 2,997. Olist issues a new customer_id on every order

What share of deliveries were late?

8.1% or 6.8%. Comparing timestamps counts on-the-day deliveries as late

A semantic layer defines each metric once, so an agent asks for revenue instead of writing its own SQL. Vendor benchmarks show this helps. What they don't separate is how much comes from the layer itself and how much from modeling it for agents: clear definitions, synonyms, stated rules. This project measures that, on real data, with questions frozen before the model was written.

Status

  • Semantic model: dbt + MetricFlow on DuckDB, 18 metrics, 30 data tests

  • Benchmark: 30 test and 8 dev questions with correct SQL, committed before the model

  • MCP server and REST API over one core, 40 tests

  • Agent page where every answer shows its definition, filters and SQL

  • Results: raw text-to-SQL vs semantic layer vs semantic layer modeled for agents

  • Hosted demo

Related MCP server: ECommerce MCP Server

Run

Requires uv.

uv sync
make data    # downloads Olist and loads it into DuckDB
make build   # builds the dbt models and runs the tests
make test    # tests the core, MCP server and REST API

Use

Three tools, the same over MCP and REST: list_metrics, get_dimensions, query_metrics. Queries name metrics, dimensions and filters; MetricFlow writes the SQL. The database is opened read-only.

Claude Code:

claude mcp add talk-to-your-metrics -- uv run --directory /path/to/talk-to-your-metrics python -m ttym.mcp_server

REST, and MCP over HTTP at /mcp:

make serve
curl -s localhost:8000/query -H 'content-type: application/json' \
  -d '{"metrics": ["revenue"], "group_by": ["metric_time__year"]}'

Data

Brazilian E-Commerce Public Dataset by Olist, CC BY-NC-SA 4.0. Downloaded by make data, never stored in this repository.

License

Code: MIT.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      Not graded
      quality
      D
      maintenance
      Enables AI agents to query live schema, lineage, and query-context across data warehouses, dbt projects, orchestration systems, and BI tools via MCP tools.
      Apache 2.0
    • F
      license
      Not graded
      quality
      D
      maintenance
      Hosted MCP server connecting Shopify, Klaviyo, GA4, Meta Ads, Google Ads, Xero, Gorgias and 20+ e-commerce data sources so AI assistants can answer merchant questions that span every source at once.
      -
    • F
      license
      A
      quality
      C
      maintenance
      Enables AI agents to discover and query enterprise knowledge documents and business metrics through standardized MCP tools, supporting retrieval, structured sales/order analysis, anomaly scanning, and data lineage lookups.
      12
      -