Skip to main content
Glama
nithin554

Code-Crawler

by nithin554

Code-Crawler

npm version npm downloads npm total downloads

Static + dynamic call tracing MCP server backed by a persistent property graph of your codebase.

Code-Crawler indexes a project into a SQLite-backed property graph β€” files, packages, modules, classes, interfaces, methods, fields, imports, annotations/decorators, call edges, virtual dispatch, aspect advice, and the wiring frameworks add (entry points, routes, injected dependencies, reflective lookups) β€” then exposes that graph to LLM clients as MCP tools: find callers, trace call paths, explore neighborhoods, follow runtime dispatch, and search symbols across the whole codebase.

Supports Java, Python, TypeScript, and JavaScript projects, and can index all of them into the same graph.

πŸ“– Documentation site: https://nithin554.github.io/code-crawler/ πŸ“˜ Task-oriented guide: USAGE.md πŸ—ΊοΈ Roadmap: ROADMAP.md

        β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€  MCP (stdio)  ───────────────────────┐
        β”‚  crawl_project  search_symbols  get_callers  get_callees     β”‚
        β”‚  trace_path     trace_execution  find_aspects  get_decoratorsβ”‚
        β”‚  find_implementations  find_hotspots  find_dead_code  ...    β”‚
        β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                        β”‚                              β”‚
                 β”Œβ”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”             β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”€β”
                 β”‚   crawler    β”‚             β”‚  graph queries  β”‚
                 β”‚ parse+resolveβ”‚             β”‚ recursive CTEs  β”‚
                 β””β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”˜             β””β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                        β”‚                              β”‚
                 β”Œβ”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”
                 β”‚        SQLite property graph (WAL)          β”‚
                 β”‚   nodes (kind, fqn, props) + edges (kind)   β”‚
                 β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Why a graph?

Plain-text grep can't answer questions like:

  • "Who transitively calls OrderRepository#create?"

  • "Is there a static path from Main#main to Tracking#dispatch?"

  • "What is the impact radius of changing OrderService#place?"

These need graph traversal. Code-Crawler builds the graph once (idempotently, file-by-file) and answers them with bounded recursive SQL queries.

Related MCP server: uni-kb

Features

  • Four languages, one graph: Java (tree-sitter-java), Python (tree-sitter-python), TypeScript (tree-sitter-typescript, tree-sitter-tsx), and JavaScript (tree-sitter-javascript) β€” all WASM, no native build step. Language is detected per file, so a polyglot repo lands in a single graph.

  • Dynamic tracing on a static graph β€” DYNAMIC_DISPATCH edges connect interface/base calls to every override in the project, and ADVISES edges weave AspectJ/Spring AOP advice into execution traces (before, after, around, after-returning, after-throwing, named @Pointcut references, &&/||/! and execution, within, @annotation, @within, target, this, args primitives).

  • Run-time indirection is named, not lost β€” Java reflection/ServiceLoader, Python getattr/importlib, and JS require/dynamic import() produce REFLECTS edges to the resolved symbol when the target is statically determinable, or to a __reflect__: marker when it is not.

  • Framework wiring β€” dependency injection (@Autowired, NestJS constructor injection, FastAPI Depends) becomes INJECTS edges, and HTTP routes (@GetMapping, @RequestMapping, NestJS @Controller/@Get, FastAPI/Flask/Quart decorators, Express app.get) become ROUTES edges to the handler, which is also marked as an entry point.

  • Control flow on call edges β€” every CALLS edge records whether the call is conditional and the compact source text of its enclosing guards/loops, so an agent can tell a happy path from an error path without reading the file.

  • External dependencies are explicit β€” third-party imports and runtime globals become package-level external_symbol nodes with REFERENCES edges and a find_external_dependencies summary, distinct from project code.

  • Annotations as first-class nodes β€” @Transactional, @Cacheable, @Async, @Retryable, Python decorators, and TS decorators become annotation nodes with ANNOTATED_WITH edges, so you can query decorators on types, methods, and fields.

  • Call resolution heuristics that go beyond naive string matching: receiver types inferred from locals, parameters, fields, new T() expressions, self/this scoping, dotted paths (service.repo.find), loop-variable element types, bare/this/super calls, static/qualified calls, and superclass chains. Unresolvable calls become unknown_symbol phantom nodes with resolved:false edges, so the graph stays traversable.

  • Incremental, file-scoped indexing β€” each crawl records a per-file manifest (size, mtime, SHA-256) and replays the cached parse result for files whose content is unchanged, so a re-crawl re-parses only what actually changed. Changes are reported as filesAdded / filesUpdated / filesUnchanged / filesRemoved. Derived dynamic edges and orphan pruning are rebuilt only when something changed. A full re-crawl still removes files that disappeared, without duplicating the graph, and scoped crawls (fileFilter or a forced language) never remove files outside their scope. When a checkout, rsync -t, an mtime-preserving unzip, or clock skew restores an edit with the old size and mtime, the shortcut misses it β€” pass force (--force on the CLI) to re-parse everything.

  • FTS5-backed symbol search β€” symbol lookup runs against an external-content FTS5 index with BM25 ranking and exact/prefix boosting, falling back to a LIKE scan only when FTS finds nothing. Search does not degrade as the graph grows.

  • Versioned schema with automatic migration β€” the database carries a schema version and upgrades itself in place on open. A database written by a newer build is rejected with a clear message instead of being read as if the shape were known.

  • A database per project β€” the index lives in <project>/.code-crawler/index.db, so two repositories can never collide. Pointing a populated database at a different project fails with a clear_index hint instead of silently wiping the first index.

  • Analytics β€” hotspots (fan-in/fan-out/advice counts) and dead-code candidates.

  • Self-contained: single SQLite file, no external services.

  • LLM-friendly: every tool returns structured JSON, unknown symbols come back with fuzzy search suggestions.

Install & build

Requires Node.js β‰₯ 22.12 (better-sqlite3@13 and the Vitest toolchain both need it).

npm install
npm run build        # tsc β†’ dist/
npm test             # vitest (full suite; set CODECRAWLER_BENCH=1 for the scale test)
npm run lint         # biome check (lint + format, fails CI)
npm run format       # biome format --write
npm run coverage     # vitest run --coverage (thresholds in vitest.config.ts)
npm run bench        # crawl/query benchmark on a generated ~137k-line project

CLI

codecrawler crawl <path>     # index a project
codecrawler crawl <path> --force  # re-parse every file, ignoring cached metadata
codecrawler status           # index summary
codecrawler query <symbol>   # fuzzy-lookup a symbol
codecrawler serve            # start the MCP server over stdio (default)

Environment variables:

Variable

Default

Purpose

CODECRAWLER_DB

<project>/.code-crawler/index.db

Database path. Resolved from the cwd upward, so each project gets its own file; falls back to ~/.code-crawler/index.db when no project encloses the cwd. The home directory is only that fallback, never an enclosing project

CODECRAWLER_EXCLUDE_DIRS

.git,node_modules,dist,build,target,...

Dirs skipped during crawl

CODECRAWLER_DEFAULT_LIMIT

50

Default result cap

CODECRAWLER_MAX_DEPTH

10

Traversal depth cap (1–100)

CODECRAWLER_MAX_FILE_BYTES

1000000

Files larger than this are skipped, not parsed

CODECRAWLER_RESPECT_GITIGNORE

true

Honor the project's .gitignore during discovery

CODECRAWLER_HELPFUL_ERRORS

true

Fuzzy suggestions on symbol misses

CODECRAWLER_PLUGIN

unset

Plugin mode: resolve one database per project at call time instead of opening one file at startup

CODECRAWLER_PLUGIN_ROOT

unset

Plugin install directory, never indexed as a project

CODECRAWLER_PLUGIN_DATA

temp dir

Writable directory for the no-project fallback database

Invalid values never crash the server: booleans fall back to their default and integers are clamped to their documented range, each with a one-line warning on stderr.

What gets indexed

crawl walks the project and skips .git, node_modules, build output, and other CODECRAWLER_EXCLUDE_DIRS entries. On top of that:

  • the project's .gitignore is honored (disable with CODECRAWLER_RESPECT_GITIGNORE=false),

  • an optional .crawlerrc in the project root can narrow or widen discovery with gitignore-style globs, and relocate where module ids start:

    {
      "include": ["src/", "packages/core/"],
      "exclude": ["**/generated/"],
      "moduleRoots": ["src"]
    }

    When include is non-empty, only matching files are indexed. A malformed .crawlerrc is ignored with a warning rather than failing the crawl.

    moduleRoots is opt-in: with ["src"], src/orders/service.ts becomes the module orders.service instead of src.orders.service, which is what a baseUrl-or-PYTHONPATH project actually imports. It is never inferred, because inferring it would silently renumber an existing index. TypeScript and JavaScript projects also get path aliases for free: compilerOptions.paths and baseUrl are read from tsconfig.json (or jsconfig.json), following extends within the project, so import { x } from "@app/orders/service" resolves to the real module instead of being recorded as external. Changing either setting re-parses cached files on the next crawl.

  • files above CODECRAWLER_MAX_FILE_BYTES and files that look binary are skipped; crawl_project reports them in skippedFiles with a reason.

  • per-file parse failures never abort the crawl; they are reported in parseErrors.

Only the root .gitignore is read β€” nested .gitignore files are not yet supported.

Incremental crawls

A crawl records a manifest row per file (size, mtime, SHA-256, language). On the next crawl a file whose size and truncated mtime both match is served from a cached parse result instead of being re-read and re-parsed; a file whose mtime changed but whose hash did not is also replayed from cache. Resolution, phantom-node creation and derived edges still run over the whole in-memory file set, so cross-file edges stay correct β€” only I/O, parsing and database writes are skipped.

crawl_project reports the split:

Field

Meaning

filesAdded

First seen by this crawl

filesUpdated

Content changed, re-parsed

filesUnchanged

Replayed from the parse cache

filesRemoved

Disappeared from disk and were pruned

Known limitation: because an unchanged file's edges are not rewritten, a call that becomes newly resolvable only because some other file changed (for example a method gained an overload, or a class was introduced that a previously-ambiguous call could now bind to) will not be upgraded in the untouched file. Touch that file, or run clear_index, to force full resolution.

Benchmarks

npm run bench generates a synthetic TypeScript project (575 modules, ~137k lines, ~9.8k methods), crawls it, and reports crawl phases and query latency percentiles. tests/perf.test.ts asserts the same budgets and is opt-in, because a realistic corpus takes time to crawl:

npm run bench                                        # human-readable report
CODECRAWLER_BENCH=1 npm test -- tests/perf.test.ts   # enforce the budgets
CODECRAWLER_BENCH_LINES=20000 npm run bench          # smaller corpus

Measured on a Windows dev laptop with a warm disk cache; absolute numbers vary, the ratios and the index usage are what matter:

Phase

Duration

Added

Updated

Unchanged

Cold crawl

22.5 s

575

0

0

Re-crawl, nothing changed

0.91 s

0

0

575

Re-crawl after a one-file edit

0.75 s

0

1

574

Query

p50

p95

Budget (p95)

search_symbols (exact)

0.69 ms

1.96 ms

50 ms

search_symbols (prefix)

1.60 ms

4.09 ms

50 ms

get_callers (depth 2)

0.16 ms

0.21 ms

50 ms

get_callees (depth 2)

0.11 ms

0.15 ms

50 ms

get_subgraph (radius 2)

0.16 ms

0.27 ms

50 ms

find_entry_points

0.12 ms

0.29 ms

50 ms

find_hotspots

145.5 ms

181.4 ms

1000 ms

find_dead_code

99.8 ms

290.3 ms

1000 ms

The index is 66 MB for those 14.4k nodes / 38.8k edges. find_hotspots and find_dead_code are whole-project aggregations over every edge and node, which is why their band is an order of magnitude looser than the symbol lookups; both were profiled with EXPLAIN QUERY PLAN and are served by idx_edges_target_kind.

CI & publishing

GitHub Actions runs on every pull request (pull-request.yml): npm ci β†’ typecheck β†’ tests β†’ build β†’ smoke-test of the built CLI.

Merges to main trigger publish.yml, which re-runs the same checks and then, if green, auto-releases:

  1. Reads the most recent v* tag (or package.json version on the first run), computes the next patch version, and pushes a vX.Y.Z tag.

  2. Creates a matching GitHub release.

  3. Publishes the package to the npm registry with provenance (OIDC trusted publishing) β€” no token or repository secret is needed. The step is idempotent and retries: it skips versions npm already has, and a transient transparency-log or registry failure no longer aborts the release.

  4. Publishes server.json to the official MCP Registry as io.github.nithin554/code-crawler from a separate, re-runnable job using ExaDev/mcp-registry-publish. The job runs inside the mcp-registry-publish environment, whose deployment branch policy is limited to main. It pins server.json to the release version, authenticates with GitHub OIDC, skips versions the registry already lists, and retries transient npm propagation or registry errors. Ownership is verified through the mcpName field in package.json.

Using as an MCP server

Install as an agent plugin

The repository is also an agent plugin, so Claude Code and Copilot CLI can install the server and the skill together:

# Claude Code
/plugin marketplace add nithin554/code-crawler
/plugin install code-crawler@code-crawler

# Copilot CLI
copilot plugin marketplace add nithin554/code-crawler
copilot plugin install code-crawler@code-crawler

The plugin runs npx -y @nithin554/code-crawler serve. Agent Plugins clients start a plugin's stdio servers inside the plugin directory, so the server does not treat that directory as your checkout: it resolves a project from crawl_project, the client's workspace roots, or the project this session already selected, and keeps that graph in <project>/.code-crawler/index.db β€” one database per repository, the same layout the CLI uses. CODECRAWLER_DB still pins a single file when you want one.

The same skill is published to ClawHub as code-crawler, and the plugin is published there too:

  • Skill β€” clawhub install code-crawler. ClawHub assigns this version by bumping the patch digit of whatever it already published, so it advances one release at a time instead of copying the npm version; a release whose skill publish fails leaves the two numbers one apart until the next publish.

  • Plugin β€” published as the code-crawler bundle plugin, versioned from the release tag: release v0.1.67 publishes the bundle at 0.1.67. The bundle's own plugins/code-crawler/package.json is packaging metadata β€” ClawHub needs a package manifest to inspect the bundle and build its ClawPack artifact, and warns when one is missing β€” so the release workflow stamps the tag's version into it, packs the bundle, and publishes that tarball; the committed version is only a placeholder.

The plugin publish runs the official clawhub CLI itself, under Node, rather than calling upstream's reusable package-publish.yml. That workflow cannot publish this bundle: it inspects the packed artifact with bun ... package validate, and the CLI refuses to run under Bun ("Plugin validation requires Node.js 22.13+ (22.x) or 24+"). A ClawPack always contains package/package.json and no input skips the step, so the call only ever succeeded while the bundle had no manifest β€” the very thing ClawHub warns about.

Publishing is restricted to main. The publishing jobs deploy through the clawhub-publish environment, whose deployment branch policy rejects any other ref, so a manual dispatch from a feature branch stops before it can upload. CLAWHUB_TOKEN is a repository secret, not an environment secret: the skill call hands it to upstream, which reads it as clawhub_token, and a job that calls a reusable workflow cannot declare an environment. Forwarding the value through a job output does not help either β€” GitHub refuses to emit a secret-valued output and logs Skip output 'clawhub_token' since it may contain secret.

One command

add-mcp registers the server in whichever client you name β€” Claude Code, Cursor, VS Code, Claude Desktop, Codex, OpenCode, Zed, and more (npx add-mcp list-agents). Run it from inside the repository you want indexed:

npx add-mcp @nithin554/code-crawler --args serve -a cursor

That writes project-scoped config next to the code (.mcp.json, .cursor/mcp.json, .vscode/mcp.json, …), so each repository carries its own registration and its own .code-crawler/index.db. The server name code-crawler is derived from the package.

--env CODECRAWLER_DB is optional β€” add it only to put the database somewhere else: --env CODECRAWLER_DB=/absolute/path/to/index.db. -g writes user-level config instead. Prefer that only for clients that launch the server with your checkout as its working directory: with -g and an unrelated working directory (Claude Desktop starts in its own install folder) the server falls back to ~/.code-crawler/index.db, and every repository then competes for that one file.

One server per project

Outside plugin mode a server process opens one database at startup and a database holds exactly one project, so the model is one registration per repository β€” which is what the per-project config above gives you. The plugin install instead resolves a database per project at call time, so one plugin serves every repository you open. Re-crawling a repository reuses its existing file and only re-parses what changed; two repositories never share a file, so they cannot clash. Only a few project databases stay open at once (idle ones are closed), and every handle is released when the client disconnects.

MCP Registry / npx

The server is listed in the MCP Registry as io.github.nithin554/code-crawler. Any npx-capable client can run it without a local checkout:

{
  "mcpServers": {
    "code-crawler": {
      "command": "npx",
      "args": ["-y", "@nithin554/code-crawler", "serve"]
    }
  }
}

Claude Desktop

claude_desktop_config.json:

{
  "mcpServers": {
    "code-crawler": {
      "command": "node",
      "args": ["/absolute/path/to/code-crawler/dist/index.js", "serve"],
      "env": { "CODECRAWLER_DB": "/absolute/path/to/your/index.db" }
    }
  }
}

Cursor / other stdio MCP clients

Point the client at the same command. The typical flow for an LLM agent:

  1. crawl_project the repo once.

  2. find_entry_points to see where execution actually starts (routes, listeners, mains).

  3. search_symbols to locate the symbol of interest.

  4. get_callers / get_callees / trace_path to reason about the call graph.

  5. trace_execution to follow runtime dispatch and aspect advice; find_aspects, get_decorators, and find_annotated to inspect behaviour-changing annotations.

  6. find_routes, find_injections, find_reflections, and find_external_dependencies to map framework wiring and third-party surface.

See USAGE.md for the full task-oriented walkthrough.

MCP surface

Tools

Tool

Purpose

crawl_project

Index a directory (idempotent; one project per database)

index_status

Node/edge/file counts + last crawl

search_symbols

Fuzzy lookup by name or FQN, optional kind filter

get_node

Metadata + source location for one symbol

get_callers

Direct or transitive callers (depth-bounded)

get_callees

Direct or transitive callees (depth-bounded)

trace_path

Static call paths between two symbols (BFS, shortest first)

trace_execution

Dynamic trace: static calls + virtual dispatch + woven advice

find_implementations

Implementers/subclasses of an interface or superclass

find_aspects

Aspects, advice kinds, pointcuts, and intercepted methods

get_decorators

Annotations/decorators on a symbol and optionally its members

find_annotated

Every symbol annotated with a given annotation

find_hotspots

Rank methods by fan-in, fan-out, and advice interceptions

find_dead_code

Methods with no incoming call/dispatch/advice edges

find_entry_points

Framework-invoked roots: HTTP routes, listeners, schedulers, CLI mains, tests

find_routes

HTTP route β†’ handler mappings (@GetMapping, NestJS, FastAPI, Express)

find_injections

DI-supplied dependencies (@Autowired, NestJS, FastAPI Depends)

find_reflections

Run-time lookups: Class.forName, getattr, importlib, dynamic import()

find_external_dependencies

Third-party packages with reference and file counts

find_unresolved

Call sites the resolver could not bind, grouped by failure reason

impact_analysis

Callers + entry points + routes + overrides + risk band in one call

explain_symbol

One-shot digest: members, decorators, callers, callees, wiring

find_cycles

Call cycles (SCCs) with a sample closed path, plus direct recursion

export_graph

Neighborhood rendered as Mermaid or Graphviz DOT

get_subgraph

Neighborhood extraction for visualization

get_schema

Describe the graph model

clear_index

Reset the database

Resources: codecrawler://schema, codecrawler://status Prompt: trace-call-path(from, to)

Token budgeting. The 18 list-returning query tools β€” every tool whose payload is a single result list β€” accept max_tokens (100–200000) and cursor. Passing either adds a page block with totalCount, returnedCount, truncated, estimatedTokens, and nextCursor; omit both and the response shape is unchanged, so existing clients keep working. Paging never returns an empty page while items remain, so a continuation loop always terminates. Token counts are estimates from JSON length (~4 characters per token) β€” no tokenizer dependency, deterministic across models.

get_subgraph is the one exception: its nodes and edges are two views of the same slice, so cutting one list would leave edges pointing at omitted nodes. It is bounded by radius and limit instead. Aggregate tools (impact_analysis, explain_symbol, export_graph) and single-object tools (get_node, index_status, get_schema) return one object and take no budget.

Agent skill

The server tells an agent what it can call; the skill tells it when. skills/code-crawler/ ships in the npm package and is plain Markdown β€” SKILL.md for discovery and the crawl-first workflow, reference/tools.md mapping all 27 tools to the questions they answer, and reference/edge-semantics.md explaining confidence, external, and phantom nodes.

npx skills add nithin554/code-crawler

The skills CLI finds it by convention and installs to whichever agents you pick; npx skills update refreshes it. To install by hand instead, copy the directory into your client's skills folder (~/.claude/skills/code-crawler/, .claude/skills/code-crawler/, or a plugin's skills/ root). No build step. See USAGE.md.

tests/skill.test.ts asserts the skill's tool table against the live registry, so it cannot drift as tools are added.

Graph model

Nodes β€” kind is one of file | package | module | class | interface | enum | method | constructor | field | import | external_symbol | aspect | annotation_type | annotation | unknown_symbol. Every node has a globally unique qualifiedName and a JSON properties blob (signature, modifiers, lines, annotations, ...).

Edges β€” kind is one of CALLS | DECLARED_IN | CONTAINS | IMPLEMENTS | EXTENDS | IMPORTS | REFERENCES | DYNAMIC_DISPATCH | ADVISES | ANNOTATED_WITH | REFLECTS | INJECTS | ROUTES. CALLS edges carry line, resolved, confidence, and β€” when the call sits behind control flow β€” conditional, controlFlow, and guards; ANNOTATED_WITH edges carry line and arguments; ADVISES edges carry adviceKind and the resolved pointcut; INJECTS carries injectedName and framework; REFLECTS carries mechanism; REFERENCES carries external.

Id scheme:

Symbol

Example

type

com.acme.OrderService

method

com.acme.OrderService#place

constructor

com.acme.OrderService#<init>

field

com.acme.OrderService$repo

annotation

@Transactional

import

__import__:java.util.List

unresolved call

__unknown__:System.out.println

Python/TS module

module:app.service

module-level function

module:app.api#read_order

module-level field

module:app.service$AUDIT_LOG

inline arrow handler

module:src.routes#<anonymous@7>

external package

ext:fastapi

reflective marker

__reflect__:./plugins/${name}

injection marker

__inject__:Repo

Dynamic behaviour

Static call edges answer "what does the source literally call?". Runtime behaviour differs in two ways, both modelled as derived edges rebuilt on every crawl:

Virtual dispatch

A call to an interface or base-class method may execute any override. After all files are indexed, every resolved CALLS edge whose target is an overridable method gets DYNAMIC_DISPATCH edges to each override found in the project:

Checkout#checkout --CALLS--> Payment#pay
Payment#pay --DYNAMIC_DISPATCH--> CardPayment#pay
Payment#pay --DYNAMIC_DISPATCH--> UpiPayment#pay

Constructors and private/static/final methods are excluded because they cannot be overridden. Use find_implementations to list candidates and trace_execution to follow them.

Aspects, advice, and decorators

@Aspect types are indexed as aspect nodes. Advice methods are recognised by @Before, @After, @Around, @AfterReturning, @AfterThrowing, and @Pointcut. Named pointcuts (@Around("paymentCall()")) are resolved to their @Pointcut expression, then matched against every project method to create ADVISES edges:

AuditAspect#auditPayment --ADVISES--> Payment#pay
AuditAspect#auditPayment --ADVISES--> CardPayment#pay
AuditAspect#auditPayment --ADVISES--> UpiPayment#pay
AuditAspect#beforeCheckout --ADVISES--> Checkout#checkout

trace_execution weaves these in as INTERCEPT hops, so the trace reads:

Checkout#checkout
  INTERCEPT  AuditAspect#beforeCheckout
  CALLS      Payment#pay
    INTERCEPT        AuditAspect#auditPayment
    DYNAMIC_DISPATCH CardPayment#pay -> authorize
    DYNAMIC_DISPATCH UpiPayment#pay   -> settle

Every other annotation is indexed as an annotation node with an ANNOTATED_WITH edge. Framework decorators (@Transactional, @Cacheable, @Async, @Retryable, @PreAuthorize, @Scheduled, ...) are additionally flagged decorator: true so get_decorators and find_annotated can surface runtime-semantic annotations.

Python decorators and TypeScript decorators are modelled the same way: @router.get(...), @injectable, @Controller(...), @Get(':id') and friends land in annotations on the declaring type/method, and framework ones additionally produce routes, injections, or entry-point marks.

Reflection and run-time indirection

Lookups the compiler cannot see through are recorded as REFLECTS edges. When the target is statically determinable β€” a string literal, a literal prefix plus a variable (importlib.import_module("app.plugins." + name)), getattr(obj, "run"), require("./x") β€” the edge points at the resolved symbol. Otherwise it points at a __reflect__: marker carrying the mechanism:

module:app.dynamic#load_plugin --REFLECTS--> __reflect__:app.plugins.   (importlib.import_module)
module:src.dynamic#loadPlugin   --REFLECTS--> __reflect__:./plugins/${name}   (import())

Dynamic module prefixes that belong to the project's own namespace stay in-project; only genuinely foreign packages become external dependencies.

Dependency injection

Dependencies the framework supplies are INJECTS edges instead of missing calls (find_injections):

Framework

Detected from

Spring

@Autowired fields/methods/constructors

NestJS

constructor parameters with decorators; @Injectable() classes

FastAPI

Depends(...) parameters

dependency-injector

constructor parameters of @injectable classes

plain TS/JS

constructor parameters typed with a project class

Primitive and structural parameter types (string, { id: string }, list[str]) are counted but produce no edge, so @Body() body: { id: string } does not pollute the graph.

Framework entry points and routes

Methods a framework invokes have no source-level caller, so marking them keeps dead-code analysis honest and gives trace_execution a root (find_entry_points, find_routes):

Language

Recognised

Java

@GetMapping, @PostMapping, @RequestMapping, @Scheduled, @EventListener, listener annotations, tests, main

Python

FastAPI (@router.get, @app.post, @router.websocket), Flask (@bp.route, @app.get), Quart (@bp.route), if __name__ == "__main__", pytest tests

TypeScript

NestJS @Controller + @Get/@Post/... , Express app.get('/x', handler), Jest tests

Express handlers written inline get a synthetic method node (module:src.routes#<anonymous@7>) so their body still gets call edges, containment, and a route edge.

Control flow on call edges

Each CALLS edge records whether it is guarded:

{
  "kind": "CALLS",
  "source": "app.repository.OrderRepository#create",
  "target": "ext:builtins",
  "properties": { "line": 14, "conditional": true, "controlFlow": ["branch", "loop"],
                  "guards": ["row.get(\"id\") == order_id", "self.rows"] }
}

controlFlow uses branch | loop | exception | predicate. Guards are source text, not symbolic constraints: they tell an agent that a call is conditional and what the condition looked like, without pretending to model its truth value.

Resolution heuristics

Java:

  1. Constructor calls (new Foo(...)) β†’ resolve Foo β†’ #<init> (default ctor if implicit).

  2. Bare / this. / super. calls β†’ enclosing type + superclass chain.

  3. Type-qualified calls (com.acme.Foo.bar) β†’ resolve the type directly.

  4. Instance calls (x.method()) β†’ infer receiver type from locals β†’ params β†’ fields, then search that type and its ancestors.

  5. Otherwise β†’ phantom unknown_symbol node, resolved:false, with the inference failure reason attached to the edge.

Python and TypeScript add module-aware resolution on top of the same pipeline:

  1. Import-aware scope β€” from app.repository import OrderRepository maps the local name to app.repository.OrderRepository; relative and project modules are resolved by suffix, bare specifiers fall back to package roots.

  2. self.x / this.x resolve against the enclosing class's fields; a receiver that starts with self/this/cls/super never becomes an external package.

  3. Dotted receivers (service.repo.find) are walked segment by segment: the first segment resolves to a type, the next to a field on it, and container members such as list.append are recognised as builtin rather than unknown.

  4. Locals are typed from parameters, annotated assignments, new T(), and inferred loop/variable element types (for row in self.rows: β†’ row: dict).

  5. Module functions (module:app.api#read_order) participate in dispatch, routes, and entry-point marking exactly like class methods.

Each resolved edge carries a confidence score and a human-readable reason trace (e.g. via local orders: OrderService), so LLM clients can weigh how much to trust it.

Implemented languages

Language

Extensions

Extracted

Dynamic features

Java

.java

packages, classes, interfaces, enums, annotations, constructors, methods, fields

virtual dispatch, AspectJ/Spring AOP advice, @Autowired, @RequestMapping family, Class.forName, ServiceLoader

Python

.py, .pyi

modules, classes, functions, methods, fields, decorators

@injectable/Depends injection, FastAPI/Flask/Quart routes, getattr, importlib, __main__, tests

TypeScript

.ts, .tsx, .mts, .cts

modules, classes, interfaces, type aliases, enums, methods, fields, arrow functions

NestJS/Angular decorators, DI constructor injection, NestJS/Express routes, import(), require()

JavaScript

.js, .jsx, .mjs, .cjs

modules, classes, functions, fields

decorators, Express routes, require()

Crawling a directory mixes all of them automatically β€” language is chosen per file from the extension, and a single crawl can return languages: ["java", "python", "typescript"].

Adding a language requires:

  1. src/parser/languages/<lang>.ts β€” a LanguageDescriptor implementing parseFile (module/package extraction, type/member extraction, raw call sites, local type tables) plus three declarative fields: resolver (which call resolver handles it), frameworks (what its parser recognises), and fixture (its conformance directory).

  2. One row in src/parser/languages/index.ts.

  3. One row in the CONTRACTS table in tests/language-conformance.test.ts β€” the suite then holds the new language to the same graph invariants as every other one.

  4. Optionally a resolver in src/resolver/ exposed through src/resolver/registry.ts; most languages work with the shared module resolver and only override language-specific builtins.

  5. Any framework it should recognise that is not registered yet β€” a module in src/parser/frameworks/ declaring its entry points, injections and markers. That is a data change: no parser is edited.

Frameworks are resolved per file by import evidence, then by the descriptor's frameworks order, then by every registered module. A file that imports dependency_injector attributes its @inject there, while a Java file with no such import still falls back to the registry rather than losing the annotation. Parsers keep what is genuinely syntactic: receiver names for call-style routes, path joining, and the text recorded as an entry point's detail.

No existing language module needs editing: dispatch in src/crawler.ts reads the descriptor's resolver, so a language never appears in a conditional.

The graph schema, store, traversal queries, and MCP surface stay the same for every language. Candidate next languages, and the modelling each one needs, are ranked in ROADMAP.md Phase 3.

What is not modelled

Language-specific gaps, in addition to the shared ones below:

Language

Not modelled

java

A record is indexed as a class with record: true; components become private final fields plus a synthesized accessor each, but the implicit canonical constructor gets no signature until a call resolves to it. Lambda bodies are indexed, but a method reference such as this::handle has no static target and produces no call edge. static {} and instance initialiser blocks are not extracted. Generic type parameters are erased, so a call through a List<T> field binds to the declared bound rather than the concrete argument.

python

No metaclass, descriptor, __getattr__, or __getattribute__ modelling. Members a decorator generates rather than writes β€” @dataclass's __init__ and __eq__, anything a metaclass adds β€” have no declaration to point at, so calls to them stay unresolved. Monkey-patching at import time is not tracked. A lambda stored on an attribute and invoked later has no target.

typescript

A namespace X {} block produces no module node, so members declared inside one are not reachable by name. JSX parses cleanly but the element tree is not modelled. Type-level constructs β€” conditional, mapped, template-literal, intersection β€” are not evaluated, so a receiver typed that way falls back to unresolved. static {} blocks and accessor fields are not extracted.

javascript

Handled by the TypeScript parser, so every gap above applies. With no annotations available, receivers come only from new, assigned initialisers, and the language-agnostic heuristics β€” dynamically typed code resolves less often than annotated code does.

Not yet modelled, by design:

  • Value flow and branch feasibility β€” edges say that a call is conditional and record the guard text, not which branch is taken.

  • Cross-repository resolution β€” libraries outside the crawled tree are summarised as ext: packages rather than crawled.

  • Fully dynamic targets (getattr(obj, name), require(userInput)) point at a __reflect__: marker instead of a symbol; there is nothing static to point at.

  • Runtime DI graphs assembled from configuration files rather than code.

  • Receiver inference from an expression rather than a name β€” getService().run(), items[0].run(), a ternary, or a cast β€” so those calls report as unresolved.

  • Overload selection uses argument count first, then the type of each argument where the call site states one (literals, new expressions, casts). Two overloads that a call cannot distinguish β€” f(count) against f(int) and f(String) β€” still bind to the first declaration in source order. Overloaded declarations keep distinct ids (Type#f, Type#f(int), Type#<init>(String)) after the first one; see USAGE.md for the id scheme.

The full list, and what is modelled in exchange for each gap, is in USAGE.md.

Architecture

src/
β”œβ”€β”€ index.ts                    CLI (crawl | status | query | serve)
β”œβ”€β”€ server.ts                   MCP server: tools, resources, prompt (thin handlers)
β”œβ”€β”€ crawler.ts                  file discovery β†’ parse β†’ index β†’ resolve β†’ persist
β”œβ”€β”€ config.ts                   env-overridable configuration
β”œβ”€β”€ graph/
β”‚   β”œβ”€β”€ schema.ts               node/edge kinds, id scheme, property shapes
β”‚   β”œβ”€β”€ store.ts                better-sqlite3 schema + idempotent file-scoped upserts
β”‚   └── queries.ts              recursive-CTE traversal (callers/callees/path/subgraph)
β”œβ”€β”€ parser/
β”‚   β”œβ”€β”€ language.ts             LanguageConfig / LanguageDescriptor contracts
β”‚   β”œβ”€β”€ treeSitter.ts           WASM runtime + grammar loading + CST walk helpers
β”‚   β”œβ”€β”€ controlFlow.ts          enclosing branch/loop/try/catch + guard text
β”‚   β”œβ”€β”€ moduleId.ts             import-aware module id derivation
β”‚   β”œβ”€β”€ methodId.ts             method id scheme (overload signatures, constructor ids)
β”‚   β”œβ”€β”€ frameworks/
β”‚   β”‚   β”œβ”€β”€ types.ts            FrameworkModule / FrameworkScope contracts
β”‚   β”‚   β”œβ”€β”€ index.ts            registry + scope/entry-point/injection lookups
β”‚   β”‚   └── <framework>.ts      one module per framework (spring, nestjs, fastapi, …)
β”‚   └── languages/
β”‚       β”œβ”€β”€ index.ts            descriptor table + per-file language detection
β”‚       β”œβ”€β”€ java.ts             Java structural extraction
β”‚       β”œβ”€β”€ python.ts           Python structural extraction
β”‚       └── typescript.ts       TypeScript, TSX and JavaScript extraction
└── resolver/
    β”œβ”€β”€ types.ts                TypeIndex (global name/method index) + Resolver contract
    β”œβ”€β”€ registry.ts             resolver lookup, capability query, default fallback
    β”œβ”€β”€ common.ts               builtin types, external packages, receiver inference
    β”œβ”€β”€ dynamic.ts              virtual dispatch + AspectJ pointcut matching/advice edges
    β”œβ”€β”€ genericResolver.ts      the `module` resolver (imports, barrels, overloads)
    └── javaResolver.ts         the `java` resolver (declared types, hierarchy, overloads)

Roadmap

Language-agnostic architecture, more languages, and better resolution are tracked in ROADMAP.md. The near-term focus is making a new language a one-module change, then using that to add Go, C#, and Kotlin.

Documentation site

The static documentation site in docs/ is deployed to GitHub Pages by .github/workflows/pages.yml. It covers quickstart, the MCP tool reference, and worked examples. Open docs/index.html locally or visit https://nithin554.github.io/code-crawler/.

License

MIT

Related MCP Connectors

Related MCP Servers

  • A
    license
    Not graded
    quality
    C
    maintenance
    Enables AI clients to perform local code search, indexing, and analysis across Java, JavaScript/TypeScript, .NET/C#, and Python projects through the MCP protocol.
    1
    Apache 2.0
  • A
    license
    Not graded
    quality
    D
    maintenance
    Enables parsing, indexing, and querying source code as structured knowledge, providing code exploration, spec generation, and migration tools via 20 MCP tools.
    MIT
  • F
    license
    Not graded
    quality
    F
    maintenance
    Provides a local code knowledge graph for Java projects, enabling querying of classes, methods, fields, calls, inheritance, and imports via MCP tools like query, context, impact, and cypher.
    1
    -