# Meridian-KG

A knowledge-graph + context-engineering layer for agentic retrieval over
governed enterprise documents, with cross-silo orchestration and an
MCP interface — built as an extension to
[Meridian](../meridian), an AI-native information management system.

## Why this exists

This project responds to two specific, verified public statements from
Waqas Ahmed, VP of AI Engineering at OpenText, given to theCUBE/
SiliconANGLE at Google Cloud Next 2026 — not a general impression of
OpenText's positioning:

> "Enterprise information is not just files on a drive. It is
> organized, governed, tagged with context, tagged with metadata and
> integrated with the business applications and customer processes.
> To wire that into the AI providers and LLMs, you have to be able to
> build that context so you are not flooding the LLMs with extra
> information, but you're giving them the right information at the
> right time."

> "If you keep the agents locked up within their individual
> applications, all you have done is automated the existing
> applications for some efficiency, but that's not the true power of
> the agentic enterprise. The true power is in the choreography and
> orchestration across silos, across applications where data provides
> the intelligence and AI executes the actions."

He also named the specific interoperability mechanism OpenText uses:
Content Aviator is exposed to other agents via agent-to-agent
integration and **Model Context Protocol**-based data connectors.

Three concrete, testable design decisions map directly to those three
statements:

| His statement | What's built |
|---|---|
| "Right information at the right time, not flooding" | Graph facts are relevance-ranked against the query and capped (`max_facts`), not dumped as a raw neighborhood traversal — see `_rank_facts` in `app/query_engine.py` |
| "Choreography and orchestration across silos" | The corpus is structured into separate silos (`legal-compliance/`, `hr/`, `finance/`), traversal is bidirectional so cross-silo dependencies surface, and every answer explicitly reports which silos it drew on |
| MCP-based interoperability | `app/mcp_server.py` exposes the query engine as real MCP tools, tested against an actual MCP client session, not just imported and assumed to work |

## What it actually does

1. **Ingests** a directory of documents, chunked with stable IDs and
   tagged by silo (department/system), so every fact traces back to
   its exact source paragraph and origin.
2. **Extracts entities and relations** via dependency parsing (no
   external LLM call required — see `app/extract.py` for a drop-in
   `LLMExtractor` stub if one is available in a given deployment).
3. **Builds a knowledge graph** (NetworkX `MultiDiGraph`) where edges
   carry the source sentence, chunk, document, and silo.
4. **Detects genuine factual conflicts** — same subject+relation,
   different values, from different documents — using lexical overlap
   between the competing values to avoid false positives.
5. **Answers queries** by combining bidirectional graph traversal
   (multi-hop, entity-anchored, relevance-ranked) with BM25 retrieval,
   surfacing conflicts explicitly and reporting which silos the answer
   drew on.
6. **Logs a full audit trail per query**, retrievable via
   `GET /audit/{query_id}` (REST) or `get_audit_trail` (MCP).
7. **Exposes everything over MCP** (`app/mcp_server.py`) so another
   agent in an orchestration pipeline can call `query_knowledge_graph`,
   `list_detected_conflicts`, or `get_audit_trail` directly — the same
   interoperability model Ahmed described for Content Aviator.

## Demo scenarios

`data/corpus/` is split into three silos:
- `legal-compliance/` — two versions of a data-retention policy that
  genuinely disagree (7 vs. 10 years for financial records; 5 vs. 3
  years for marketing consent), plus org-roles and incident-response
  docs
- `hr/` — an employee offboarding policy that notifies the compliance
  team about departing employees with financial data access
- `finance/` — a vendor payment policy that also notifies the
  compliance team, independently, about reconciliation issues

Querying the retention conflict returns both values with sources and
an explicit non-resolution note. Querying "who does the compliance
team report to" resolves through two hops (chief privacy officer →
general counsel) — a fact no single sentence states. Querying "what
does the compliance team get notified about" pulls facts from all
three silos in one answer and reports that explicitly — the
choreography-across-silos scenario.

## Running it

```bash
pip install -r requirements.txt
python -m spacy download en_core_web_sm   # or install the wheel directly, see below

# run the test suite (38 tests, including live API and live MCP protocol tests)
pytest tests/ -v

# run the REST API
uvicorn app.main:app --reload
# POST /ingest
# POST /query   {"question": "How long does the company retain financial records?"}
# GET  /graph/conflicts
# GET  /audit/{query_id}

# run the MCP server (for another agent/client to connect to via stdio)
python -m app.mcp_server
```

If `pip download` of the spaCy model is blocked by network policy, the
model wheel can be installed directly:
```bash
pip install https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.8.0/en_core_web_sm-3.8.0-py3-none-any.whl
```

## Test coverage

38 tests across ingestion, extraction, graph construction, conflict
detection, retrieval, the query engine, the live REST API (via
`TestClient`), and the live MCP server (via a real in-memory MCP
client session — `mcp.shared.memory.create_connected_server_and_client_session`,
not just importing the module). Real bugs were caught and fixed during
development, not just after the fact:

- **False-positive conflicts**: the first version of conflict
  detection flagged any subject+relation pair with multiple objects,
  which incorrectly flagged unrelated facts sharing a verb. Fixed by
  requiring lexical overlap between the competing values — see
  `_same_fact_template` in `app/graph.py`.
- **Non-idempotent re-ingestion**: calling ingest twice (once at API
  startup, once via a manual `POST /ingest`) silently doubled every
  edge in the graph because the `MultiDiGraph` was never reset. Fixed
  in `MeridianKGSystem.ingest_corpus`.
- **Multi-hop facts crowded out by same-hop noise**: naive keyword
  overlap ranking scored same-hop facts that just repeat the anchor
  entity's name higher than genuinely relevant multi-hop facts whose
  wording doesn't mention the anchor. Fixed with weighted scoring that
  favors relation/target-word matches over anchor-word repeats.
- **Bidirectional traversal fanning out into unrelated chains**:
  expanding "both directions" at every hop caused hop-2 traversal to
  wander into HR's and Finance's own unrelated reporting chains once
  they entered the frontier via an incoming edge. Fixed by only
  continuing forward (out-edge) chains past hop 1.
- **`from __future__ import annotations` breaking FastMCP tool
  introspection**: this MCP SDK version's tool-schema generation
  can't resolve stringified type hints, so it raised a `TypeError` at
  import time. Fixed by removing the future import in `mcp_server.py`.
- **List-returning MCP tools silently mis-parsed by clients**:
  FastMCP 1.9.4 serializes a `list[dict]` return as one content block
  per list item rather than one JSON array, so naive client code
  (`json.loads(result.content[0].text)`) only sees the first item.
  Fixed by having list-returning tools return an explicit JSON string
  instead, so the output shape is predictable to any calling agent.

## Honest limitations

- Entity/relation extraction is rule-based (spaCy dependency parsing),
  not LLM-based. The `LLMExtractor` interface in `app/extract.py` is
  there so swapping to an LLM extractor is a drop-in change, not a
  rewrite.
- Conflict detection is a lexical-overlap heuristic, not semantic
  understanding — it will miss conflicts phrased very differently
  across documents. This is a genuinely open problem industry-wide;
  a 2026 workshop paper on knowledge graphs for context engineering
  found the same scalability and quality-consistency gaps.
- The graph store is in-memory (NetworkX), fine at this scale, not a
  substitute for a production graph database at enterprise document
  volumes.
- Silo separation here is a directory convention (`legal-compliance/`,
  `hr/`, `finance/`), not a real multi-system integration — it
  demonstrates the orchestration *pattern* Ahmed described, not an
  actual connection to separate live systems.
- This has not been benchmarked against IDOL, Aviator, or AIDP
  directly — those aren't publicly testable at this scope. The
  comparison being made is conceptual (what the architecture
  demonstrates in response to specific public statements), not a
  head-to-head performance claim.
- The two SiliconANGLE quotes above are the only fully-verified,
  word-for-word statements from Ahmed used to inform this build.
  LinkedIn post content referenced earlier in this project's research
  was never actually read (LinkedIn requires login) — only post
  titles were visible via search, so no design decision here is based
  on unverified LinkedIn content.

## Architecture

```
data/corpus/{legal-compliance,hr,finance}/*.txt
      │
      ▼
app/ingest.py        chunking + stable IDs + silo tagging + provenance
      │
      ├──────────────┬─────────────────┐
      ▼               ▼
app/retrieval.py  app/extract.py  (entities + SVO triples, silo-tagged)
  (BM25)               │
      │                ▼
      │          app/graph.py  (KG build + conflict detection + bidirectional traversal)
      │                │
      └────────┬───────┘
               ▼
      app/query_engine.py   (relevance ranking, cross-silo detection, audit trail)
               │
       ┌───────┴────────┐
       ▼                ▼
app/main.py       app/mcp_server.py
(FastAPI REST)    (MCP tools: query_knowledge_graph,
                   list_detected_conflicts, get_audit_trail)
```