Ultron

Evidence-first binary and firmware analysis.

Ultron sits on top of mature engines — Ghidra headless, binwalk — and contributes the thing they do not: a project model where every finding is a structured claim linked to the exact artifacts that support it.

Free-text security claims are not merely discouraged here. They are unrepresentable. A claim is a registered predicate with typed fields, and it cannot be stored without artifact ids of the kinds that predicate demands. An agent that tries to write "this looks exploitable" gets a schema error naming the offending field.

Ultron builds no disassembler and no decompiler, and it never will. That work is already done well; the gap is everything around it.

Formerly named Aether, then Yugen. Yugen (幽玄) is a Japanese aesthetic term for a truth perceived through subtle suggestion rather than full disclosure — the same principle behind evidence-only, never-over-assert claims, and the name this project carried while it was developed standalone. It is now renamed Ultron on joining a personal multi-agent developer ecosystem, where each specialist tool takes an agent name; here it is the reverse-engineering and security specialist. The rename is cosmetic — every invariant below, the evidence-graph discipline, the claim schema, and the zero-dependency core, carries over unchanged. See ADR 0011 for the full reasoning.

Status: Phase 0 released, Phase 1 and 2 landed, Phase 3 partial

All 25 gate checks pass, 385 tests pass.

python examples/demo_phase0.py

  [PASS] ELF identified with architecture and word size
  [PASS] functions, xrefs, and decompilation imported
  [PASS] Ghidra converged onto existing artifacts instead of duplicating
  [PASS] nested container chain unpacked (uImage -> gzip -> cpio)
  [PASS] findings attributed to the member file, not the container blob
  [PASS] claim resolves to a string artifact at a concrete location
  [PASS] free-text claim from an agent is refused
  [PASS] two independent analyses produce byte-identical graphs
  [PASS] suite 'elf_sample' passes  -  recall 1.00, 0 false positive(s)
  ...
  25/25 gate checks passed

Phase 0 is released as v0.1.0-phase0.

Phase 1 has landed: a narrow natural-language interface and light emulation-based reachability.

$ ultron ask "are there any hardcoded secrets?"
[hardcoded_secrets] 11 claim(s) cited

11 credential-shaped literals across the project: 3 private key,
3 connection string, 2 aws access key, 1 api token, 1 ssh authorized key.
  [clm_0217e368bbeb, clm_f2f751393fbc, clm_1284ca2d2406, +8 more]
etc/telemetry.conf carries api token, connection string, password.
  [clm_1284ca2d2406, clm_0853d39c8248, clm_16dc579a703b]
  caveat: These are pattern matches. A match is credential-shaped; whether it
          is a live credential requires a human to check.

Five question types, matched deterministically with no language model involved and no network request. Anything outside the set is declined with the supported list, never guessed at. Every explanatory line cites the claim ids it rests on, enforced by validate_answer rather than merely intended (ADR 0006).

Measured on a seventy-case labelled corpus: accuracy 1.00, macro precision 1.00, zero false accepts. The corpus states its own limitation - vocabulary and cases share an author, so this measures internal consistency rather than performance against phrasings nobody anticipated.

All 22 Phase 1 checks pass, covering both deliverables end to end:

python examples/demo_phase1.py

  [PASS] question set is narrow, as specified (4-5 types)
  [PASS] every explanatory line cites the claims it rests on
  [PASS] an unsupported question is declined, not guessed at
  [PASS] execution is refused unless explicitly allowed
  [PASS] a function that never ran produces no reachability claim
  [PASS] the attack-surface answer now reports observed execution
  [PASS] question classification suite passes
  ...
  22/22 Phase 1 checks passed

Phase 2 has landed, as four narrow, real capabilities rather than the full "cartography and campaigns" surface at once:

$ ultron map
[cartography] run run_b05687d13748b3bad67b684d986c0f8e
  8 file(s) considered, 1 link(s) found

consumer   symbol             provider           conf
---------  -----------------  -----------------  ----
bin/app    EVP_EncryptUpdate  lib/libcrypto.so   0.6

$ ultron diff-versions ../previous-build --record
kind     component  from      to
-------  ---------  --------  ------
changed  openssl    1.0.2u    3.0.1

recorded 1 component_version_changed claim(s)

Cross-binary import/export linking resolves which file's undefined symbol is satisfied by which other file's exported one, by name - a name match, not proof of a live dependency, and the resulting claims say so at reduced confidence. Version diffing compares embedded components across two independently analysed projects and can record what changed, evidenced in the newer project.

$ ultron reach
[cartography] run run_d61b9bbf020e8572570e084d717f686d
  3 observed function(s) considered, 2 cross-binary sink(s) reached

observed in            function          reaches symbol  conf
----------------------  ----------------  --------------  ----
bin/firmware_agent      handle_name       strcpy          0.90
bin/firmware_agent      run_diagnostics   system          0.95

$ ultron diff-graph ./baseline ./current
comparing baseline -> current
  artifacts: +12 -0   claims: +8 -0

Cross-binary sink reachability chains three claims that already independently exist - a function observed executing (QEMU), a call site into an import (Ghidra), and that import resolved to another file's export (ultron map) - into a claim that a specific, observed code path reaches the boundary of another binary at a named symbol. Its confidence is the minimum, not the product, of the two chained claims: this is one reasoning chain, not two independent observations corroborating each other.

The general evidence-graph diff turned out to be nearly free given content-addressed ids (ADR 0002): comparing two graphs is a set difference over ids, so ultron diff-graph compares any two projects, two exports, or a project against its own export.

examples/demo_phase2.py covers the first two capabilities end to end (14/14 checks); reachability and the graph diff are exercised by their own dedicated suites, tests/test_reachability.py and tests/test_graph_diff.py:

python examples/demo_phase2.py

  [PASS] a matching export resolves the import
  [PASS] a ubiquitous libc symbol (malloc) is excluded by default
  [PASS] a name-match join scores below a direct header reading
  [PASS] re-running linking converges rather than duplicating
  [PASS] the same linking is available to agents via ultron_map
  [PASS] a version bump between two projects is detected
  [PASS] the recorded claim cites real evidence, not just text
  [PASS] the baseline project is never written to
  [PASS] 'added' components are never written as version-change claims
  ...
  14/14 Phase 2 checks passed

Campaign/fleet correlation groups several project directories - not artifacts within one project - by shared evidence: an identical file (same SHA-256) is conclusive; several shared component-version pairs is a weaker signal, gated by a threshold so one common library is never mistaken for a lineage.

$ ultron campaign ./firmware-v1 ./firmware-v2 ./unrelated-device
campaign 1: ./firmware-v1, ./firmware-v2
    ./firmware-v1 <-> ./firmware-v2  (shared_file: 1 identical file(s), e.g. bin/bootloader (f4e90c13...))

no correlation found for: ./unrelated-device

All four items the specification names under "Firmware Cartography & Campaigns" now have a landed, scoped slice - see ADR 0008 for exactly where each line was drawn, including an honest limitation the component-fingerprint signal surfaced against this repository's own test fixtures.

Phase 3, partially: the approval-workflow half of "Richer Agents & Expansion" is deterministic, local, and fits the existing evidence model exactly - a claim's status field and the fact that an agent-submitted claim already lands as proposed. That part is built:

$ ultron review list
id                predicate        conf  producers   subject  statement
----------------  ---------------  ----  ----------  -------  ------------------------
clm_6baf38592005  contains_string  0.50  agent:demo  bin/app  {"text": "sketchy string"}

$ ultron review approve clm_6baf38592005 --reviewer neil --note "reviewed, looks fine"
clm_6baf38592005... -> accepted by neil

ultron_review_queue lets an agent see the queue over MCP, but there is no approve or reject MCP tool, and there never will be - a test enumerates the tool registry and asserts no tool name contains "approve" or "reject". An agent can see whether its own proposal is still pending; it cannot close the loop itself. See ADR 0009.

The first specialist agent has landed, narrowly: ultron agent secrets sends string evidence already in a project to a locally-running LLM (Ollama) and proposes contains_hardcoded_secret / suspicious_string claims for patterns the deterministic rules would plausibly miss. Its backend is local-only by hard requirement, not a default - no cloud vendor is ever called - and, like every other agent submission, its output lands as proposed and needs a human running ultron review approve to go further. There is no MCP tool for it, on purpose, for now. See ADR 0010.

$ ultron agent secrets --model llama3.2 --max-claims 5
[agent secrets] proposed 1 claim(s)
  considered 42   skipped: existing 6, low-confidence 3, malformed 0
  these are 'proposed', not 'accepted' - review them with 'ultron review list' and 'ultron review approve'

This is one agent, not "full specialist agents" in the general sense the specification meant - other specialist agents, and exposing this one over MCP, remain future, separate decisions.

Broader Accessible Mode is the one Phase 3 item still not attempted, deliberately: it means loosening the deliberately narrow, measured question set from Phase 1 (ADR 0006), whose whole point was that narrow is what makes precision reproducible. That is a choice for the project's owner to make explicitly, not a default to assume.

What works today

The two "not yet run" rows are stated plainly because they matter. The translation layers on both sides are fully tested; what has not executed is the subprocess invocation, because neither engine is installed on the machine Phase 0 was built on. See Enabling the full engines.

Quick start

Python 3.10+ and no runtime dependencies. Nothing to install:

git clone https://github.com/n-3-0-l-d-3-v/aether-platform.git
cd aether-platform
python examples/demo_phase0.py

Working with a project directly:

python cli/ultron.py init ./work
python cli/ultron.py -P ./work analyze examples/demo_firmware.bin
python cli/ultron.py -P ./work query objects
python cli/ultron.py -P ./work query claims --predicate contains_hardcoded_secret
python cli/ultron.py -P ./work query claim clm_1284ca2d2406
python cli/ultron.py -P ./work export ./work/export

Sample binaries are generated, not committed. examples/demo_phase0.py and the test suite build them on demand; to build them by hand:

python examples/src/build_elf_sample.py examples/firmware_agent.elf
python examples/src/build_firmware_sample.py examples/demo_firmware.bin

Installing puts ultron on PATH:

pip install -e .
ultron doctor

Running the tests

python -m pytest              # 385 tests
python -m pytest -q tests/test_evidence_model.py   # the invariants alone

The suite generates its own sample binaries on first run. PE-specific tests skip cleanly on hosts that cannot produce a PE - note that a native gcc on Linux compiles the sample into an ELF, so presence of a compiler is not enough and the output is checked for an MZ header. Install a mingw-w64 cross-compiler for PE coverage on Linux.

CI runs the suite, the gate demonstration, an export-determinism check, and the evaluation suites on Linux, Windows, and macOS across Python 3.10 and 3.12.

What it looks like

$ ultron analyze demo_firmware.bin
[binwalk] run run_e65361591a1e...
  engine ultron-carver   extracted 7 file(s)
    bin/diagnostics.exe                    pe          132.4 KiB
    bin/firmware_agent                     elf         1.8 KiB
    etc/dropbear/dropbear_rsa_host_key.pem certificate 196 B
    etc/telemetry.conf                     data        219 B

$ ultron query claims --predicate contains_hardcoded_secret
id                predicate                  conf  prod  ev  subject             statement
----------------  -------------------------  ----  ----  --  -----------------  --------------------
clm_1284ca2d2406  contains_hardcoded_secret  0.95  1     1   etc/telemetry.conf  {"detector": "rul...
clm_0217e368bbeb  contains_hardcoded_secret  0.98  1     1   etc/dropbear/dro..  {"detector": "rul...

$ ultron query claim clm_1284ca2d2406
claim   clm_1284ca2d2406f45deb3f680afb7914f5
schema  ultron.claim.contains_hardcoded_secret/1
stated  {"detector": "rule:github-token", "redacted_preview": "ghp_****", "secret_kind": "api_token"}
conf    0.95 (max 0.95 across 1 producer(s))

evidence
role   kind    addr  artifact          name
-----  ------  ----  ----------------  ----------------------------------------
locus  string  0x56  art_6a75100355c5  api_key=ghp_A1b2C3d4E5f6G7h8I9j0K1l2...

Every finding walks back to bytes. That is the entire point.

The model

Three record types carry everything.

Artifact — a concrete, locatable piece of evidence: a file, a function, a string, an xref, a section, a decompiled body, a signature hit. Its id is a hash of its identity fields only, so enriching an artifact never changes its id, and two engines observing the same thing land on the same row.

Claim — a structured assertion: a registered predicate, typed fields, and the artifacts backing it in named roles (locus, support, context, counter). It carries no producer and no timestamp, so the same assertion from two engines is one claim.

Attestation — one producer standing behind one claim at one moment, with a confidence. Confidence is never a property of a claim; it is derived from the attestations — maximum within a producer, noisy-OR across independent producers. Two engines agreeing at 0.9 gives 0.99, not two near-duplicate findings.

That split is the central design decision: ADR 0003.

What is enforced, not merely encouraged

Free text has exactly one home: annotations, in their own table and their own export stream, where they can never be mistaken for findings.

Enabling the full engines

Ultron runs without Ghidra or binwalk, at reduced depth, and says so. ultron doctor reports every component - including the JDK on its own row, because Ghidra headless fails on a missing or too-old runtime in a way that reads as a Ghidra problem - along with what each gap costs and how to close it:

$ ultron doctor
ultron 0.1.0  (python 3.12.2, win32)

  ok       triage    0.1.0      built in; no external engine required
  MISSING  java      -          not found on PATH or under JAVA_HOME
           cost:     Ghidra headless cannot start at all.
           fix:      Install a JDK 21 or newer (for example Temurin, from
                     https://adoptium.net) and put 'java' on PATH, or set
                     JAVA_HOME.

  MISSING  ghidra    -          analyzeHeadless was not found
           cost:     No function recovery, cross references, or decompilation.
                     Header-level triage still runs.
           fix:      Install Ghidra (https://ghidra-sre.org) and set
                     GHIDRA_INSTALL_DIR to its directory, or put
                     support/analyzeHeadless on PATH. Or skip the local install
                     entirely: ultron import-ghidra <dir> --object <file>
                     ingests an export produced on any machine.

  MISSING  binwalk   -          binwalk was not found on PATH
           cost:     squashfs, jffs2, ubifs, and vendor formats are located but
                     not unpacked. gzip/bzip2/xz/zip/tar/cpio still work.
           fix:      Install it with 'pip install binwalk', or from
                     https://github.com/ReFirmLabs/binwalk. Full extraction
                     also wants sasquatch, jefferson, and ubi_reader.

1 of 4 components available.
Ultron still runs: header triage, firmware carving, the evidence graph,
the MCP server, and export all work without any external engine.

Ghidra

Provides function recovery, cross references, decompilation, and precisely located strings. Without it, header-level triage still runs.

1. Install Ghidra (11.x recommended).

2. Install a JDK 21 or newer and make sure java is on PATH, or set JAVA_HOME. Ghidra headless will not start without it.

3. Point Ultron at the install:

   export GHIDRA_INSTALL_DIR=/opt/ghidra_11.1.2_PUBLIC     # Linux/macOS
   setx GHIDRA_INSTALL_DIR "C:\ghidra_11.1.2_PUBLIC"       # Windows

   ULTRON_GHIDRA_HOME and GHIDRA_HOME are also honoured, and support/analyzeHeadless on PATH works too. Failing all of those, Ultron checks the conventional install directories.

4. Verify and run:

   ultron doctor
   ultron -P ./work analyze ./target.elf --engine ghidra

You do not need Ghidra locally to use Ghidra results. The bridge splits running from importing, so an export produced on any machine can be ingested anywhere:

# on the machine that has Ghidra
analyzeHeadless /tmp/proj ultron -import target.elf \
    -scriptPath ultron/adapters/ghidra/scripts \
    -postScript UltronExport.py /tmp/export 40 "" -deleteProject

# anywhere
ultron -P ./work import-ghidra /tmp/export --target ./target.elf

UltronExport.py runs inside Ghidra's own interpreter (Jython 2.7, or CPython under PyGhidra) and stays in the subset both accept.

binwalk

Provides squashfs, jffs2, ubifs, and vendor formats. Without it, the built-in carver handles gzip, bzip2, xz, zip, tar, and cpio, and reports anything it could only locate rather than silently skipping it.

pip install binwalk
# or: https://github.com/ReFirmLabs/binwalk

Full extraction also wants sasquatch, jefferson, and ubi_reader, which are awkward on Windows — the reason the fallback carver exists (ADR 0005).

Asking questions

A deliberately narrow interface: five question types, and anything else is declined rather than answered badly.

ultron ask --list                              # what it answers
ultron ask "what third-party components are in this image?"
ultron ask "is this binary hardened?" --object bin/busybox
ultron ask "what is the attack surface?"

The same surface is available to agents as the ultron_ask MCP tool, returning the identical structured record. Why no language model: ADR 0006.

Reachability

Which recovered functions actually executed - what separates "this binary imports system()" from "and run_diagnostics ran".

qemu-user is an emulator, not a sandbox. Its system calls pass through to the host kernel, so a traced binary can do anything a native process could. Ultron never executes a target implicitly: ultron analyze will not do it, and ultron trace requires --allow-execution. Use a disposable VM or container. See ADR 0007.

# on an isolated machine
qemu-arm -d exec,nochain -D trace.log ./target

# anywhere - executes nothing
ultron import-trace trace.log --object bin/target

Recording and importing are split exactly as they are for Ghidra. The parser handles -d exec and -d in_asm, prefers the former because its counts are real execution counts, and infers the load base of a position-independent binary - declining to guess when the alignment is unconvincing, so an unaligned trace produces no claims rather than wrong ones.

MCP

The MCP server is the interface future agents work against, and it is a peer of the CLI — both are thin front ends over one library.

ultron mcp              # stdio JSON-RPC
ultron mcp --read-only  # hide and refuse every write tool

Twenty-one tools: inventory, artifact and claim queries, string search, decompilation retrieval, graph traversal, schema discovery, provenance, ultron_ask for the question interface, plus ultron_submit_claim and ultron_annotate for writes. Agent-submitted claims go through exactly the validation an adapter does and land as proposed.

The transport is the hand-rolled JSON-RPC-over-stdio implementation from ADR 0004, not the mcp PyPI package - kept that way through the rename so the zero-runtime-dependency guarantee (ADR 0001) still holds; see ADR 0011 for why that was a deliberate choice rather than an oversight.

The ecosystem agent contract

Ultron is one specialist tool in a personal multi-agent developer ecosystem (ADR 0011). Three pieces exist purely for that:

• agent.yaml at the repo root - a static manifest (name, role, sensitivity tier, entrypoint, health-check command, vault path) a future orchestrator reads without parsing this project's source.

• ultron --health - prints a small, stable JSON block (version, whether Ghidra/binwalk are reachable, last-run timestamp, cached test-suite status) for a future health-polling agent. Deliberately separate from ultron doctor, which stays human-facing prose with wrapped remedies.

• ultron vault write/list/approve - writes RE findings as Markdown notes into vault/Ultron/pending/. Nothing here auto-approves a note; only a human running ultron vault approve moves one into vault/Ultron/approved/, where it counts as committed knowledge. This is a separate gate from ultron review: review approves structured claims already inside a project's evidence graph, while the vault holds prose findings meant for a knowledge base outside any one project - free text that the schema system was never meant to check.

The private sensitivity tier

agent.yaml declares default_sensitivity_tier: private: no analysis data leaves this machine. That was already true by convention (ADR 0010's local- only LLM backend); it is now also checked at construction time by ultron/network_policy.py - pointing the secrets-triage agent's backend at anything but localhost/127.0.0.1 raises RemoteHostRefused unless ULTRON_ALLOW_REMOTE_AGENT_HOST=1 is set explicitly. Every other module in this project performs no network I/O at all; tests/test_ecosystem.py patches socket.socket.connect and urllib.request.urlopen and runs init/analyze/ask/--health underneath the patch to check that claim structurally, not just by inspection.

Git-friendly export

ultron export writes two trees, and the split is the point:

• graph/ — artifacts, claims, links. Content-addressed, sorted by id, no timestamps or run ids. Two independent analyses of the same bytes produce byte-identical files. Commit this; the diff shows what was discovered.

• ledger/ — runs, attestations, observations. Provenance is a record of events, so it grows. That is correct.

Evaluation

Ground truth lives in eval/suites/*.json:

$ ultron eval
[PASS] elf_sample       required 22/22   recall 1.00   false positives 0
[PASS] firmware_image   required 12/12   recall 1.00   false positives 0

An expectation can demand a confidence floor, a minimum number of independent producers, and — importantly — that the matched claim cites evidence of a specific kind. A contains_hardcoded_secret claim pointing at a file rather than a string fails, even though the statement reads identically.

Recall is a real figure because a suite can enumerate what must be found. Precision is scored only against explicitly forbidden patterns, since no suite can enumerate everything true about a binary; unexpected claims are reported as unscored volume rather than folded into a flattering number. The harness has negative controls in the test suite — a harness that cannot fail proves nothing.

Layout

ultron/
  canonical.py       deterministic serialization, hashing, id minting
  agents/            specialist agents: local-only LLM backend, secrets triage
  evidence/          artifact kinds, claim predicates, and their invariants
  project/           SQLite schema, migrations, and the only sanctioned store
  adapters/
    triage/          ELF/PE headers, strings, rule-based detectors
    ghidra/          headless runner, export script, importer
    binwalk/         firmware unpacking with a standard-library fallback
    qemu/            user-mode reachability tracing and trace parsing
  cartography/       cross-binary linking, reachability, diffing, campaigns
  export/            deterministic JSONL export, general graph diff
  mcp/               stdio MCP server and its tool surface
  nl/                the narrow question interface and its answer model
  review/            human approval workflow over agent-submitted claims
  eval/              evaluation harness
  health.py          JSON status for --health, for an ecosystem health poller
  vault.py           RE findings as reviewed Markdown notes (vault/Ultron/)
  network_policy.py  structural enforcement of the 'private' sensitivity tier
cli/                 entry point runnable without installing
docs/                architecture and decision records
eval/suites/         ground truth
examples/            sample generators and the gate demonstration
tests/               385 tests
agent.yaml           ecosystem manifest: role, sensitivity tier, health check

Documentation

• Architecture — layers, data model, and why each piece is shaped the way it is

• Decision records — the choices where a reasonable engineer would ask "why that way?":

  • 0001 Python for the core, zero runtime dependencies

  • 0002 Content-addressed ids, and what is excluded from them

  • 0003 Claims and attestations are separate records

  • 0004 The MCP server speaks the protocol directly

  • 0005 A bounded extraction fallback when binwalk is absent

  • 0006 The NL interface contains no language model

  • 0007 Emulation never runs implicitly

  • 0008 Phase 2 lands as four narrow, real capabilities

  • 0009 Approval and rejection are never exposed as MCP tools

  • 0010 Specialist agents are local-only and CLI-only

  • 0011 Rename to Ultron, joining a personal multi-agent ecosystem

A note on the sample data

examples/src/ generates binaries containing deliberately fake credentials — AWS's own published example key (AKIAIOSFODNN7EXAMPLE), a synthetic ghp_A1b2C3d4... token, PEM headers with no key material, and joke passwords. None of it is real, and none of it is live. It exists so the evaluation suite has a target whose ground truth is known exactly.

Licence

Apache-2.0.