Skip to main content
Glama
shaumikp26

orgintel

by shaumikp26
README.md
# orgintel

An MCP server that analyzes Salesforce **permission architecture** — profiles,
permission sets, and permission set groups — to support RBAC remediation. It reads an
org, snapshots the permission model into a local DuckDB store, and answers questions
about it. **Phases 1–4** (acquisition, analysis core, eval harness, agent).

Two hard guarantees, both enforced as mechanism rather than policy:

- **Read-only.** orgintel never issues an insert, update, delete, or metadata deploy
  against any org. Enforced by a transport-layer route allowlist (see
  [`clients/transport.py`](src/orgintel/clients/transport.py)); a non-allowlisted
  request raises before it is sent.
- **No personal identifiers persisted.** The `users` table has no name, username, or
  email column — the columns do not exist. `soql_query` masks identifier columns in its
  results. See [DESIGN-P1.md](DESIGN-P1.md) §2.7.

## Why it's built this way

A mid-size org has ~850k `FieldPermissions` rows — a raw dump is ~20M tokens, ~100× a
context window. **The model never sees raw API output.** Every tool returns an
aggregate, a diff, or a bounded slice; the join happens in DuckDB. Every response
carries a `budget` that discloses truncation and how to narrow.

## Setup

Requires [`uv`](https://docs.astral.sh/uv/) and the Salesforce CLI (`sf`).

```bash
uv sync
uv run pytest        # 51 tests, no org needed
```

## Auth (Phase 1: SFDX token reuse)

orgintel reads the access token from an org you've already authenticated with the
Salesforce CLI — zero extra setup. JWT bearer flow is a planned seam, not yet built.

```bash
sf org login web --alias my-org      # once
sf org list                          # confirm it's Connected
```

The subprocess argv is fixed and its output (which contains a live token) is never
logged.

## Tools

| Tool | What it returns |
|---|---|
| `snapshot_org(org_alias, scope?)` | Bulk-fetches the permission model into DuckDB. Returns **summary stats only** — counts, timing, snapshot_id. |
| `list_snapshots()` | Available snapshots, newest first. |
| `list_profiles(snapshot_id)` | Profiles by user count: license, perm cardinality, ModifyAllData flag, unassigned count. |
| `describe_object(org_alias, api_name)` | Live field list with a `sensitivity_guess` flag (SSN/DOB/account-number/… from config), sensitive fields first. |
| `permission_diff(snapshot_id, principal_a, principal_b)` | Symmetric difference of two principals. Differing fields grouped by `(a_perm, b_perm)` signature. The workhorse. |
| `soql_query(org_alias, query, max_rows=200)` | Read-only SOQL. SELECT-only parser guard, row cap, identifier columns redacted. |
| `find_sensitive_access(snapshot_id)` | **(P2)** Every principal granting read/edit on a name-matched sensitive field, with affected user counts resolved through effective permissions. Recall-first. |
| `verify_decomposition(snapshot_id, proposal)` | **(P2)** Formal check that a proposed base-profile + permission-set/group refactor leaves every user's effective permissions unchanged. Returns per-user added/removed grants; an uncovered user fails. |
| `propose_decomposition(snapshot_id, threshold?, target_base_count?)` | **(P4)** Computes a thin-base + permission-set decomposition (deterministic) and runs it through `verify_decomposition` before returning. If it doesn't preserve every user's access, you get the failure, not the proposal. Returns placeholder keys + member names + shared grants for the model to name. |

`scope` defaults to all tables. Apex in `SetupEntityAccess` is excluded by default
(84% of that table, ~noise for RBAC); pass `"setup_entity_access:apex"` to include it.

Two P2 functions — `effective_permissions` (union across profile + permission sets + PSG
components, minus muting) and `cluster_profiles` (Jaccard clustering of profiles) — are
internal building blocks, not yet exposed as tools. `verify_decomposition` is the crown
jewel: permission-refactor correctness is *formally checkable*, so "did the refactor
preserve access" is a boolean, not a judgement call — and `propose_decomposition` is
gated by it, so an unverifiable proposal is never returned.

## Agent (P4)

`agent/` is an MCP client that drives these tools via the Anthropic API
(`claude-opus-5`). The division of labor is the whole point: **the tools compute and
verify; the model orchestrates, names, and judges.** `propose_decomposition` returns a
*verified* structure with placeholder names; the model names each base in the customer's
vocabulary and writes the rationale. The model also judges whether an unmatched field
name looks sensitive (`mbr_num__c`, `dob_enc__c`) — the gap P3 measured as evasive recall.

```bash
uv sync --extra agent
orgintel-agent "snapshot <orgalias>, then propose a decomposition"   # billable
```

The model call sits behind an `LLM` protocol, so the loop, tool bridge, and judge are all
tested with a scripted fake — **no API key, no billing**. Only `orgintel-agent` makes real
calls. `uv run python -m evals.agent_delta` measures the evasive-recall lift the judge
buys (0.0 → 1.0 with the offline stand-in; detectable recall held at 1.0).

## Register with Claude Desktop

Add to `~/Library/Application Support/Claude/claude_desktop_config.json`:

```json
{
  "mcpServers": {
    "orgintel": {
      "command": "uv",
      "args": ["run", "orgintel"],
      "cwd": "../MCP Salesforce"
    }
  }
}
```

Then in Claude: *"Snapshot my-org, then show me the profiles with the most users, then
diff the top two."*

## Eval harness (P3)

The highest-signal artifact in the project: a scoreboard for the deterministic analysis
core, built *before* the agent so it develops against numbers, not vibes.

```bash
uv run python -m evals.run      # writes evals/REPORT.md + a run to evals/history/
```

`fixtures/generate.py` synthesizes org snapshots with **planted ground truth**
(deterministic given a seed): latent roles, redundant pairs that should collapse,
adversarial "do not consolidate" cases, and sensitive fields — some pattern-detectable,
some named to *evade* (`mbr_num__c`, `dob_enc__c`). The harness scores cluster purity,
sensitive-field recall (detectable target 1.0; evasive recall is the gap the P4 model
must close, measured explicitly), and privilege preservation (`verify_decomposition` must
certify a ground-truth decomposition and catch a broken one). Each run diffs against the
previous, so a prompt or pattern change that drops recall shows up immediately.

## Snapshot store

One DuckDB file, default `~/.orgintel/snapshots.db`, override with `ORGINTEL_DB`. Set it
per client engagement to keep each org's data in its own file. A snapshot holds
principal/object/field metadata and **pseudonymous** user ids — re-identification
requires authenticated access to the source org. Snapshot files are git-ignored.

## Layout

```
src/orgintel/
  clients/     thin async SF wrappers (REST, Bulk 2.0, SFDX auth) — no DB, guarded transport
  store/       DuckDB schema (SQL migrations), ingest, queries, snapshot loader — no network
  analysis/    pure functions: diff, effective perms, clustering, sensitivity, verify, propose — no I/O
  agent/       MCP client + Anthropic-backed model (LLM protocol), tool loop, field judge
  config/      sensitivity.yaml (patterns + setup-entity scope)
  snapshot.py  the acquisition coordinator (auth -> fetch -> ingest)
  server.py    nine FastMCP tools — thin
fixtures/      deterministic synthetic orgs with planted ground truth (P3)
evals/         scoreboard: run.py (deterministic core) + agent_delta.py (P4 model lift)
```

See [DESIGN-P1.md](DESIGN-P1.md) for the schema rationale and [CLAUDE.md](CLAUDE.md) for
conventions.

TDQS

A4.2/5.0

Scored across 9 tools

Disambiguation5/5

Each tool targets a distinct operation: snapshots, listing, describing, querying, profiling, diffing, sensitive access search, and decomposition verification/proposal. No overlap in functionality.

Naming Consistency5/5

All tools follow a consistent verb_noun snake_case pattern (e.g., snapshot_org, list_snapshots, permission_diff), making the purpose clear from the name.

Tool Count5/5

9 tools is well-scoped for the domain of Salesforce org permission analysis and decomposition. Each tool serves a clear role without redundancy.

Completeness5/5

The toolset covers the entire workflow: fetching permission architecture, describing objects, querying, listing profiles, diffing permissions, finding sensitive access, and proposing/verifying decompositions. No obvious gaps.

Maintenance

ActivitySlowing
ResponsivenessNo issues