Skip to main content
Glama
JoaoAndrade18

MCP Observability Server

POC — MCP server + client + REST API (observabilidade)

Stack completo e executável: uma API REST de observabilidade, um servidor MCP que a expõe como ferramentas, e um cliente MCP sem LLM que roda um roteiro de investigação real.

O dataset é sintético mas tem uma história plantada: checkout-api v2.4.0 subiu há 3 horas e quebrou o serviço. A ferramenta de correlação precisa encontrar isso sozinha — e encontra.

Para o porquê de cada decisão de design, veja docs/ARCHITECTURE.md.

Rodar

Requer uv e Python 3.14+.

git clone https://github.com/JoaoAndrade18/mcp-server-client-poc.git
cd mcp-server-client-poc
uv sync

./run.sh          # sobe tudo, roda o cenário, derruba
uv run pytest -q  # 18 testes

Related MCP server: MCP Observability Server

Arquitetura

   cliente MCP  ──MCP/streamable-http──►  servidor MCP  ──HTTP+Bearer──►  API REST
   (sem LLM)         :8765/mcp              (adapter)        :8081        (dados)
                                                 │
                                            /metrics  /healthz

As três camadas são separadas de propósito:

  • A API REST (api.py) não sabe o que é MCP. É o "sistema que já existe" na sua empresa. O servidor MCP é um adaptador na frente dela, não uma reescrita.

  • O servidor MCP (mcp_server.py) não tem dados. Traduz protocolo e guarda o token da API. Quem fala MCP nunca vê essa credencial.

  • O cliente (client.py) não tem modelo. Tudo que um LLM faria dinamicamente, ele faz por script. Se quebrar, o problema é seu servidor — não o raciocínio do modelo. É o jeito de isolar as duas coisas.

O que o POC demonstra

Área

Onde

Tools com output estruturado (Pydantic)

mcp_server.py — 6 tools

Annotations (readOnlyHint, destructiveHint)

idem — só open_incident escreve

Resources (estáticos + template)

obs://services, obs://service/{name}/runbook

Prompts (workflow reutilizável)

investigate_service

Auth por token + escopos

StaticTokenVerifier, _require_scope

Métricas por tool (calls/erros/latência)

GET :8765/metrics

Health check com upstream

GET :8765/healthz

Erros acionáveis

query_logs("does-not-exist")

Tools

Tool

Tipo

O que faz

list_services

read

Catálogo com time, tier e SLOs

query_logs

read

Busca por serviço, nível, substring, janela

get_metrics

read

Resume 1 métrica + veredito de SLO

get_deploys

read

Histórico de deploys

correlate_deploy_with_errors

read

Antes/depois do último deploy + veredito + confiança

open_incident

write

Abre incidente (exige escopo incidents:write)

correlate_deploy_with_errors é o ponto principal do design: as tools são moldadas por tarefa, não por endpoint. Ela responde "por que X quebrou?" numa chamada só, em vez de obrigar o chamador a costurar get_deploys + 2×get_metrics + query_logs e fazer a aritmética sozinho.

Rodar as partes separadas

# API REST isolada (docs em /docs)
uv run uvicorn obs.api:app --port 8081

# Servidor MCP — HTTP (produção)
uv run python -m obs.mcp_server

# Servidor MCP — stdio (Claude Desktop, IDEs)
MCP_TRANSPORT=stdio uv run python -m obs.mcp_server

# Inspector oficial
uv run mcp dev src/obs/mcp_server.py

Modo autenticado

Por padrão a auth vem desligada para o demo rodar sem fricção. Para ligar:

MCP_REQUIRE_AUTH=1 uv run python -m obs.mcp_server
MCP_TOKEN=mcp_oncall_token uv run python -m obs.client

Comportamento verificado:

Token

list_services

open_incident

nenhum

HTTP 401

HTTP 401

inválido

HTTP 401

HTTP 401

mcp_readonly_token (obs:read)

❌ bloqueado no escopo

mcp_oncall_token (+incidents:write)

Os tokens estáticos são só do POC. Em produção, StaticTokenVerifier vira um verificador de JWT contra o JWKS do seu IdP — o formato do retorno (AccessToken com escopos) é o mesmo.

Nota sobre o /metrics

O p95 do servidor foi o que revelou um bug real durante a construção deste POC: correlate_deploy_with_errors reportava p95 abaixo da média, o que é impossível. A causa era int(n * 0.95) - 1, que com n=2 dá índice 0 e retorna o mínimo. Está corrigido em percentile() e travado por teste.

É o argumento para instrumentar o servidor MCP desde o dia 1: sem o /metrics o bug teria passado.

Estrutura

src/obs/
  data.py        dataset determinístico (seed fixa, incidente plantado)
  api.py         API REST — FastAPI, bearer auth, /metrics
  mcp_server.py  servidor MCP — tools, resources, prompts, auth, métricas
  client.py      cliente MCP — sem LLM, roteiro fixo
tests/test_poc.py
run.sh

Available Tools

6 tools
correlate_deploy_with_errorsA
Read-only

Find whether a recent deploy explains a service's current errors.

Compares error rate and p99 latency before vs. after the most recent deploy in the window, pulls matching ERROR log samples, and returns a verdict with a confidence level.

This is the tool to reach for on "why is X broken" — it does in one call what would otherwise be a get_deploys + two get_metrics + query_logs chain, and it does the before/after arithmetic for you.

ParametersJSON Schema
NameRequiredDescriptionDefault
serviceYes
lookback_minutesNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
serviceYes
verdictYes
confidenceYes
p99_after_msYes
p99_before_msYes
p99_delta_pctYes
sample_errorsYes
suspect_deployYes
error_rate_afterYes
error_rate_beforeYes
error_rate_delta_pctYes

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate readOnlyHint=true, and the description adds substantial behavioral detail: it 'compares error rate and p99 latency before vs. after', 'pulls matching ERROR log samples', and 'returns a verdict with a confidence level.' This goes beyond the annotations and helps the agent understand what the tool actually does internally and what kind of output to expect.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Every sentence earns its place: purpose, behavior, and usage guidance are all compressed into three short paragraphs. It is front-loaded with the main purpose and avoids irrelevant details. The structure is clean and efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given an output schema exists (which covers return structure), the description is complete enough. It covers the analysis logic, the window concept, and when to use the tool. The complexity of a multi-step correlation is well summarized, and the agent has enough context to invoke the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It refers to 'the window', which implicitly maps to the lookback_minutes parameter, and the schema shows a required 'service'. However, it does not explicitly explain the parameter names, types, or constraints beyond the vague 'window' mention. The parameter names are self-explanatory but the description adds minimal semantic value.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Find whether a recent deploy explains a service's current errors.' The verb 'find' and resource 'recent deploy vs. service errors' are specific. It distinguishes itself from siblings like get_deploys and query_logs by focusing on correlation and before/after arithmetic.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicit when-to-use guidance is provided: 'This is the tool to reach for on "why is X broken"' and it contrasts with the alternative chain of get_deploys + get_metrics + query_logs. This gives the agent clear criteria for selecting this tool over siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_deploysA
Read-only

Deploy history, newest first. Omit service for the whole fleet.

ParametersJSON Schema
NameRequiredDescriptionDefault
serviceNo
since_minutesNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare read-only, and the description adds value by specifying the ordering (newest first) and the behavioral effect of omitting `service` (fleet-wide). It does not describe the time-based filtering or return format, but for a read-only history tool, this is reasonable coverage beyond annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short, front-loaded sentences: the first states the core purpose and ordering, the second provides a critical parameter hint. Every word earns its place, with no filler or restating of the tool name.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers the core behavior and fleet scoping, and the output schema handles return values. However, it entirely omits the time-filter parameter (`since_minutes`), leaving agents unaware of the default 24-hour window or how to filter by time. Given the tool's simplicity, this gap is noticeable but not fatal.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema descriptions are 0% for both parameters. The description explains `service` by indicating how to use it for fleet-wide results, but it says nothing about `since_minutes`, which the schema only defines with type and constraints. The name 'since_minutes' is somewhat self-explanatory, but the description fails to compensate for the lack of schema descriptions for all parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: retrieving deploy history, ordered newest first. It also distinguishes this tool from siblings by focusing on deploys and mentioning the 'whole fleet' scope, which aligns with the tool's name and differentiates it from services, logs, metrics, and incidents.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides direct guidance on using the `service` parameter: omit it for the whole fleet. This clarifies a key usage decision. However, it does not explicitly mention alternatives or when to use this instead of sibling tools like `correlate_deploy_with_errors`, so it's clear context without exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_metricsA
Read-only

Summarise one metric for one service over a window.

Returns min/mean/p95/latest plus whether the service's SLO is breached, so you get the judgement and not just the numbers.

ParametersJSON Schema
NameRequiredDescriptionDefault
metricNolatency_p99_ms
serviceYes
since_minutesNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
maxYes
p95Yes
meanYes
latestYes
metricYes
serviceYes
verdictYes
slo_breachedYes
window_minutesYes

TDQS

A4/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description reveals that the tool returns min/mean/p95/latest and SLO breach status, adding valuable context beyond the readOnlyHint and openWorldHint annotations. It also conveys the tool's judgment-focused behavior ('so you get the judgement and not just the numbers'), which helps the agent understand the tool's output character. No contradictions with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences, front-loaded with the core purpose. The second sentence adds concise and useful output details without redundancy. Every sentence earns its place, making it highly efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool has 3 parameters and an output schema, the description covers the essential purpose and return value highlights. It doesn't mention edge cases, error scenarios, or relationships to sibling tools, but for a read-only metric summarizer, it is largely complete. Slight gap in usage context prevents a 5.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It maps the parameters conceptually: 'one metric' → metric, 'one service' → service, 'over a window' → since_minutes. However, it doesn't elaborate on allowed values or the meaning of the metric enum, leaving the schema names to carry the load. This is adequate but not fully compensating for the lack of parameter descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action: 'Summarise one metric for one service over a window.' It distinguishes itself from sibling tools by focusing on metrics and SLO status, but it doesn't explicitly name alternatives like query_logs or correlate_deploy_with_errors, so it misses the top score.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied through the purpose statement, but the description does not provide explicit guidance on when to choose this tool over its siblings, nor does it mention any exclusions or prerequisites. It merely describes what the tool does, leaving the when-to-use context to the agent.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_servicesA
Read-only

List every monitored service with its team, tier and SLO targets.

Call this first when you don't already know a service name — the other tools reject names that aren't in this list.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the readOnlyHint annotation, the description discloses that this is the authoritative source for service names and that sibling tools validate against this list. This is useful behavioral context for the agent to plan its workflow.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise and front-loaded: the first sentence states exactly what the tool does, and the second sentence adds essential usage guidance. Every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple list tool with no parameters and an output schema present, the description fully covers purpose, usage, and the key constraint of being the discovery entry point. It is contextually complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, so the description need not explain parameter semantics. The baseline for no parameters is 4, and the description correctly implies no inputs are required.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('List') and resource ('every monitored service') and clarifies the data returned (team, tier, SLO targets). It is clearly distinct from sibling tools that focus on logs, metrics, deploys, and incidents.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly tells the agent to call this tool first when a service name is unknown, and warns that other tools reject names not in this list. This provides clear when-to-use guidance and distinguishes it from alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

open_incidentA

File an incident. This pages the on-call engineer — use it deliberately.

Requires the incidents:write scope; read-only callers are rejected. Prefer confirming with a human before calling this: it is the one tool here with a real-world side effect.

ParametersJSON Schema
NameRequiredDescriptionDefault
titleYes
serviceYes
severityYes

Output Schema

ParametersJSON Schema
NameRequiredDescription

No output parameters

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description goes well beyond the annotations, disclosing that it pages the on-call engineer, requires a non-read-only scope, and rejects read-only callers. This adds important behavioral context about side effects and access control that the annotations alone do not convey.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is only three sentences, opens with the primary action, and then provides essential usage warnings. Every sentence serves a purpose; no filler or redundant content.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple action tool with an output schema, the description fully covers the key context: it states the purpose, the real-world side effect, the required scope, and the need for human confirmation. Nothing important is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 0% description coverage and the description provides no explanation of the parameters or how to choose severity. The parameter names are somewhat self-explanatory, but the description does not compensate for the lack of schema-level documentation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

"File an incident" is a specific verb + resource, clearly identifying the action. It stands out unambiguously from the read-only sibling tools like list_services and query_logs.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives explicit guidance: "use it deliberately" and "Prefer confirming with a human before calling." It also notes the incidents:write scope requirement and that it is the only tool with a real-world side effect, strongly signaling when it should—and should not—be used.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

query_logsA
Read-only

Search a service's logs within a time window.

Use level="ERROR" to skip the noise, and contains to grep for a substring (a trace id, "timeout", an error code). Returns newest first.

ParametersJSON Schema
NameRequiredDescriptionDefault
levelNo
limitNo
serviceYes
containsNo
since_minutesNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and openWorldHint=false, so safety is covered. The description adds behavioral context: logs are searched within a time window and returned newest first. It does not contradict annotations and provides concrete ordering/scoping behavior beyond the safe-read label.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise—three short sentences—and front-loaded with the primary purpose. The second sentence offers actionable tips, and the third states ordering. No filler or redundant restatement of the tool name.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the output schema exists and annotations cover read-only safety, the description provides sufficient context for a log-search tool. It explains the core use case, useful filtering strategies, and result ordering. The sibling list makes it clear where this tool fits. Missing details like limit defaults are already present in the schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description carries the burden. It explains level (filter to ERROR), contains (grep substring), and time window (since_minutes) with concrete examples like trace ID and timeout. It does not explicitly mention limit, but the schema's default/min/max make that self-evident.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: 'Search a service's logs within a time window.' This clearly distinguishes it from sibling tools like list_services, get_metrics, and get_deploys, which target other resource types. The time-window scope and 'returns newest first' detail further clarify the tool's role.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives practical guidance: using level="ERROR" to skip noise and contains to grep for substrings like trace IDs or error codes. It implies this tool should be used for log investigation, which contrasts with the metrics/deploy/incident siblings, though it does not explicitly name an alternative or exclusion.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 6 tool updatesv0.1.0
    • First observedcorrelate_deploy_with_errors
    • First observedget_deploys
    • First observedget_metrics
    • First observedlist_services
    • First observedopen_incident
    • First observedquery_logs

TDQS

A4.4/5.0

Scored across 6 tools

Disambiguation5/5

Each tool has a clearly distinct purpose: service discovery, log search, metric retrieval, deploy history, deploy-error correlation, and incident creation. There is no overlap between these functions, and the correlator tool explicitly bundles several lower-level operations, reducing potential confusion.

Naming Consistency5/5

All tool names follow a consistent verb_noun pattern in lowercase snake_case (list_services, query_logs, get_metrics, get_deploys, correlate_deploy_with_errors, open_incident). The verbs are specific to each action, and the naming style is uniform across the set.

Tool Count5/5

Six tools is well within the ideal range for a focused observability server. Each tool earns its place by covering a distinct aspect of monitoring and operations, without unnecessary bloat or missing essential capabilities.

Completeness4/5

The set covers the core observability workflow: discovering services, inspecting logs, fetching metrics with SLO status, reviewing deploys, and opening incidents. A minor gap is the lack of incident management features beyond creation (e.g., listing or updating incidents), but this is secondary to the server's primary monitoring purpose.

Maintenance

ActivityStale
ResponsivenessNo issues

Related MCP Connectors

Related MCP Servers

  • F
    license
    Not graded
    quality
    C
    maintenance
    Exposes DevOps/SRE operations like Kubernetes pod status, Prometheus metrics queries, and deploy history as tools for MCP-compatible clients.
    -
  • F
    license
    Not graded
    quality
    C
    maintenance
    MCP server for observability that provides tools for log search, metrics inspection, SQL querying, incident summaries, and service discovery.
    -
  • F
    license
    Not graded
    quality
    C
    maintenance
    Enables AI agents to investigate production incidents by exposing service health, logs, and deployment data through MCP tools.
    10 npm
    -
  • A
    license
    Not graded
    quality
    B
    maintenance
    MCP server that provides guarded, audited, read-only access to ops tooling (alerts, metrics, logs, deploys, runbooks) and a triage agent that diagnoses incidents end-to-end with CI-verified root cause analysis.
    MIT