catalog-context-graph

Turn a Backstage software catalog into an operational context graph that AI agents query over MCP.

An on-call agent asked "orders-db is down — what breaks and who do we page?" needs a dependency walk, not a document search. catalog-context-graph loads your catalog-info.yaml files (plus optional OpenTelemetry service-graph edges and incident history) into a typed graph and exposes eight bounded, read-only tools over the Model Context Protocol: blast radius, ownership and escalation, dependencies, paths, change risk, and catalog drift. Neo4j is an optional system of record.

catalog-info.yaml ─┐
OTel service graph ┼─▶ typed graph ─▶ bounded tools ─▶ MCP (stdio / streamable HTTP) ─▶ agent
incident history ──┘        │
                            └─▶ Neo4j (optional: system of record, Bloom/Browser exploration)

Why a graph, measured

The usual alternative is "RAG over the catalog": embed or index each entity's YAML and retrieve the top-k for the question. The repository ships a deterministic evaluation of that baseline on the example catalog (ccg eval): 16 operational questions, ground truth computed from declared relations, and BM25 top-k retrieval scored by recall. The recall measure is deliberately generous to the baseline — an answer entity counts as covered if its name appears anywhere in the retrieved text, whether or not the chain connecting it to the question is there.

One-hop questions are easy for retrieval. Everything past the first hop — which teams to page, which storefront component fails when a database two layers down does — is where flat retrieval loses the answer, even before a model has to reason over it. Full output: eval/results-acme.json. Limitations are below; the questions are in eval/questions.yaml.

Related MCP server: Context Catalog MCP

Quick start

pip install "catalog-context-graph[mcp] @ git+https://github.com/ajinb/catalog-context-graph@main"

ccg stats --catalog path/to/catalog/                     # entity and relation counts
ccg query --catalog path/to/catalog/ blast_radius resource:orders-db
ccg query --catalog path/to/catalog/ owners checkout-service
ccg query --catalog catalog/ --service-graph sg.json catalog_drift
ccg serve --catalog path/to/catalog/                     # MCP over stdio

Try it on the bundled example (44 entities: 4 systems, 8 teams, 11 components, 8 resources, 4 APIs):

ccg query --catalog examples/acme/catalog-info.yaml \
  --service-graph examples/acme/service-graph.json \
  --incidents examples/acme/incidents.json catalog_drift

{
  "undeclared_calls": [
    "component:checkout-service -> component:fraud-scoring",
    "component:order-worker -> component:payments-service"
  ],
  "unobserved_dependencies": ["component:order-worker -> component:email-service"]
}

MCP client configuration

{
  "mcpServers": {
    "catalog": {"command": "ccg", "args": ["serve", "--catalog", "/path/to/catalog"]}
  }
}

Tools

Every tool is annotated readOnlyHint and closed-world, is depth- and size-capped, and takes entity references — never query text.

Security model

• No model-written queries. There is no "run this Cypher" tool. Agents get eight bounded questions; a prompt-injected agent can at most ask one of them about a different entity.

• Refs are validated against the Backstage reference grammar before any lookup; anything else is rejected.

• The Neo4j backend uses constant, parameterized statements. Entity text is only ever a parameter value; labels and relationship types come from a fixed allowlist. Tests assert both.

• Read-only by construction. The server holds the graph in memory and has no write path.

Inputs

• Backstage catalog — catalog-info.yaml files or directories (multi-document YAML). Reads owner, system, domain, dependsOn, providesApis, consumesApis, subcomponentOf, group parent and user memberOf. Annotations used: cloudandsre.com/tier, cloudandsre.com/runbook, pagerduty.com/service-id, opentelemetry.io/service-name.

• Service graph — {"edges": [{"client", "server", "requests"}]}, the shape the OpenTelemetry Collector servicegraph connector, Tempo and Jaeger can be exported to. Adds observed calls edges; unknown services are reported, not guessed.

• Incidents — [{"id", "title", "date", "affects": [refs]}].

Neo4j

pip install "catalog-context-graph[neo4j]"
export NEO4J_PASSWORD=...
ccg neo4j-push --catalog catalog/ --service-graph sg.json --uri bolt://localhost:7687
ccg serve --neo4j-uri bolt://localhost:7687        # serve from Neo4j as the system of record

Entities become :Entity nodes with a kind label (:Component, :Resource, :API…); relations become DEPENDS_ON, CONSUMES_API, CALLS and so on. The agent-facing tools run on the graph read back from Neo4j, so they behave identically on either backend.

Limitations

• The evaluation uses a small hand-built catalog and a lexical (BM25) baseline. Dense-embedding retrieval and end-to-end LLM answer accuracy are not measured yet; both are on the roadmap.

• Graph traversal is in memory. That is comfortable for catalogs of thousands of entities; much larger catalogs need server-side traversal in Neo4j.

• change_risk is a transparent heuristic, not a calibrated model. Its factors are returned so a human can disagree with them.

Roadmap

• Live Backstage catalog API ingestion (in addition to files)

• Dense-retrieval baseline and an LLM answer-accuracy eval mode

• Server-side Cypher traversal for very large catalogs

• Ingest Kubernetes ownership labels and Terraform state as additional edges

Related

Part of the cloudandsre.com toolkit: mcp-schema-drift (lock the tool surface an agent sees), alert-explainer, cloudandsre-skills.

License

Apache-2.0. See CITATION.cff to cite this software.