TekAutomate MCP Server

This server is the AI orchestration layer for TekAutomate. It accepts full workspace context from the app, runs tool-assisted reasoning (or deterministic shortcuts), validates output, and returns applyable ACTIONS_JSON.

What it does

• Hosts AI chat endpoint used by TekAutomate (/ai/chat).

• Proxies OpenAI Responses API with a server-owned key and vector store (/ai/responses-proxy).

• Loads and indexes local project knowledge.

• public/commands/*.json (SCPI command truth source).

• public/rag/*.json (retrieval chunks).

• public/templates/*.json (workflow examples).

• mcp-server/policies/*.md (behavior and output constraints).

• Exposes a tool catalog for retrieval, validation, and optional live instrument probing.

• Applies post-check and repair logic before returning final text.

• Stores request/debug artifacts for diagnostics.

High-level flow

1. TekAutomate sends POST /ai/chat with:

• userMessage

• provider/model/key

• full flowContext (steps, backend, model family, selected step, validation state)

• full runContext (logs/audit/exit code)

• optional instrumentEndpoint (code executor + VISA resource)

2. MCP server runs runToolLoop(...):

• deterministic shortcut path when eligible

• or provider path (OpenAI hosted Responses/tool loop, OpenAI chat-completions fallback, Anthropic)

3. Post-check validates and normalizes response:

• ACTIONS_JSON structure

• step schema and IDs

• saveAs presence/deduplication

• SCPI verification pipeline

• prose truncation guard

4. Server returns JSON payload:

• text

• displayText

• openaiThreadId

• errors

• warnings

• metrics

Endpoints

• GET /health

• returns { ok: true, status: "ready" } when indexes are loaded.

• GET /ai/debug/last

• returns last debug bundle (prompts, timings, tool trace metadata).

• POST /ai/chat

• main orchestration endpoint for TekAutomate assistant.

• POST /ai/responses-proxy

• streaming Responses proxy using OPENAI_SERVER_API_KEY and optional COMMAND_VECTOR_STORE_ID.

• POST /ai/key-test

• validates provider/key/model reachability (openai or anthropic).

• POST /ai/models

• lists available models for given provider/key.

Tooling surface

Server tools are grouped into retrieval, materialization, validation, and live-instrument calls.

• Retrieval tools:

• search_scpi

• get_command_group

• get_command_by_header

• get_commands_by_header_batch

• search_tm_devices

• retrieve_rag_chunks

• search_known_failures

• get_template_examples

• get_policy

• list_valid_step_types

• get_block_schema

• Materialization tools:

• materialize_scpi_command

• materialize_scpi_commands

• finalize_scpi_commands

• materialize_tm_devices_call

• Validation tools:

• validate_action_payload

• validate_device_context

• verify_scpi_commands

• Live instrument tools (via code executor):

• get_instrument_state

• probe_command

• get_visa_resources

• get_environment

Deterministic shortcut features

The server includes shortcut builders for common requests to produce fast, consistent actions without full model/tool loops when conditions match.

• Measurement shortcut (including scoped channel handling and standard measurement sets).

• FastFrame shortcut for pyvisa flows.

• Common pyvisa server shortcut for frequent setup/build patterns.

• tm_devices measurement shortcut.

• Planner-driven deterministic shortcut from parsed intent + command index.

These shortcuts still pass through post-check before response.

Safety and output enforcement

• Strict action schema validation (validate_action_payload).

• Replace-flow hardening.

• ensures step IDs are present and unique

• can auto-group long flat flows into logical groups

• enforces/repairs query saveAs

• deduplicates save names

• SCPI verification and source-backed command handling.

• Python substitution guard in non-python flows.

• Response prose truncation guard (MCP_POSTCHECK_MAX_PROSE_CHARS, default 1200).

• Prompt/policy driven constraints loaded from:

• mcp-server/prompts/*.md

• mcp-server/policies/*.md

Data and indexes

At startup, the server initializes:

• Command index (public/commands/*.json)

• tm_devices index

• RAG indexes (public/rag/*.json)

• Template index (public/templates/*.json)

Command sources include modern and legacy scope families plus AFG, AWG, SMU, DPOJET, TekExpress, and RSA datasets.

Frontend integration

Current app integration resolves MCP host from:

• localStorage["tekautomate.mcp.host"]

• or REACT_APP_MCP_HOST

• fallback: http://localhost:8787 only on localhost app hosts

Example:

localStorage.setItem('tekautomate.mcp.host', 'http://localhost:8787');

Run locally

npm install
npm run start

Default port is 8787 unless MCP_PORT is set.

Deploy on Railway

This repo is ready to deploy directly as a standalone Node service.

Recommended Railway settings:

• Root directory: repo root

• Install command: npm install

• Start command: npm start

• Health check path: /health

Included deploy helper:

• railway.json

Useful hosted endpoints after deploy:

• /

  • simple browser status page with uptime and links

• /health

  • JSON health payload for Railway and monitoring

• /status

  • same JSON status payload for manual checks

Suggested Railway environment variables:

• OPENAI_SERVER_API_KEY

• COMMAND_VECTOR_STORE_ID (if using file search / vector retrieval)

• OPENAI_ASSISTANT_ID (if using assistant-thread routing)

• MCP_ROUTER_ENABLED=true (if you want router-backed tool hydration at boot)

• NODE_ENV=production

Environment variables

Copy .env.example to .env and set what you need.

• Required for /ai/responses-proxy:

• OPENAI_SERVER_API_KEY

• Optional retrieval augmentation:

• COMMAND_VECTOR_STORE_ID

• OpenAI routing/model controls:

• OPENAI_BASE_URL

• OPENAI_DEFAULT_MODEL

• OPENAI_FLOW_MODEL

• OPENAI_REASONING_MODEL

• OPENAI_ASSISTANT_MODEL

• OPENAI_MAX_OUTPUT_TOKENS

• Hosted prompt controls:

• OPENAI_PROMPT_ID

• OPENAI_PROMPT_VERSION

• legacy fallback accepted: OPENAI_ASSISTANT_ID

• Prompt file overrides:

• TEKAUTOMATE_STEPS_INSTRUCTIONS_FILE

• TEKAUTOMATE_BLOCKLY_INSTRUCTIONS_FILE

• Post-check tuning:

• MCP_POSTCHECK_MAX_PROSE_CHARS

• Server:

• MCP_PORT

Scripts and verification

• npm run start / npm run dev

• npm run eval:comprehensive

• npm run eval:levels

• npm run verify:command-groups

Reference benchmark:

• mcp-server/reports/level-benchmark-2026-03-18.md shows 40/40 PASS in that run.

Logs and debug artifacts

• Last debug state: GET /ai/debug/last

• Request logs are written under mcp-server/src/logs/requests (rotated, max 500 files).

• Additional logs and reports are under mcp-server/logs and mcp-server/reports.

Internals: Planner, Materializers, and AI Routing

Intent planner (src/core/intentPlanner.ts)

The planner is a deterministic parser + resolver layer used before (and sometimes instead of) LLM output.

Main responsibilities:

• Parse user text into structured intent fields (channels, trigger, measurements, bus decode, acquisition, save/recall, status, AFG/AWG/SMU/RSA).

• Detect device type and map request to relevant command families.

• Resolve concrete SCPI candidates against the command index.

• Return unresolved intents when command mapping is ambiguous.

• Run conflict checks (resource collisions / inconsistent intent combinations).

Core exported functions:

• parseIntent(...): builds PlannerIntent from natural language.

• planIntent(...): parse + resolve + conflict check, returning PlannerOutput.

• resolve*Commands(...): domain resolvers such as resolveTriggerCommands, resolveMeasurementCommands, resolveBusCommands, resolveSaveCommands, etc.

• parse*Intent(...): focused parsers such as parseChannelIntent, parseTriggerIntent, parseMeasurementIntent, parseBusIntent, etc.

SCPI source of truth (src/core/commandIndex.ts)

• Loads command JSON files from public/commands once at startup.

• Normalizes heterogeneous command shapes (manual-entry rich format and flat format).

• Builds fast lookup structures for:

• exact header lookup (getByHeader)

• prefix lookup (getByHeaderPrefix)

• ranked query search (searchByQuery)

• Supports placeholder-aware header normalization (CH<x>, MEAS<x>, BUS<x>, {A|B}, etc.).

Current local index size (measured):

• ~9307 normalized command records.

SCPI retrieval functions

• search_scpi (src/tools/searchScpi.ts): query search + header-like direct matching merge.

• get_command_by_header: exact deterministic match for known headers.

• get_commands_by_header_batch: batch exact lookup for multiple headers.

• get_command_group: feature-area retrieval (group-level).

• verify_scpi_commands (src/tools/verifyScpiCommands.ts): validates commands (including exact syntax mode).

Materializers

Materializers convert canonical records into concrete, applyable commands/calls.

• materialize_scpi_command:

• selects set/query syntax

• infers placeholder bindings from concreteHeader

• applies explicit bindings + argument values

• checks unresolved placeholders

• runs exact verification before returning success

• materialize_scpi_commands: batch wrapper around single materializer.

• finalize_scpi_commands: batch materialize + verified output packaging, used as endgame tool in hosted flows.

• materialize_tm_devices_call: builds exact Python call from verified methodPath and arguments.

Tool loop and when server goes to AI for more info

Routing is centralized in src/core/toolLoop.ts.

Deterministic path first (no external model):

• In mcp_only mode, server tries deterministic shortcuts and planner synthesis first.

• If planner fully resolves commands, it can return applyable ACTIONS_JSON directly.

• If unresolved in mcp_only, server returns findings/suggested fixes instead of calling external AI.

AI path (mcp_ai and normal hosted usage):

• If deterministic path is not enough, server calls provider path:

• OpenAI hosted Responses (preferred for structured build/edit)

• OpenAI chat-completions fallback

• Anthropic messages path

• For hosted structured build, server preloads source-of-truth context via tools (search_scpi, get_command_group, get_commands_by_header_batch, or search_tm_devices) before/within the loop.

• Tool rounds are capped (4 for hosted structured build, 3 default, 8 when forced tool mode).

Reliability fallbacks after AI response:

• Post-check pass validates and normalizes output.

• If model returns non-actionable output, server attempts hybrid planner gap-fill.

• If ACTIONS_JSON is malformed, server retries once with strict JSON-only instruction.

• If model output is weak in specific cases, server can fallback to deterministic shortcut output.

Performance snapshot

From checked-in benchmark report:

• mcp-server/reports/level-benchmark-2026-03-18.md: 40/40 PASS.

• In that run, per-case totalMs ranged from about 1ms to 254ms.

Local micro-benchmark (quick developer run on this workspace; indicative, not production SLA):

• searchByQuery average: ~0.54 ms per lookup.

• getByHeader average: ~0.009 ms per lookup (hot path).

• materializeScpiCommand average (single-command path with verification): ~25.4 ms.

• finalizeScpiCommands average for 3-command batch: ~1.8 ms.

Use these as practical engineering baselines; real end-to-end latency depends more on provider/model calls than local index lookup.

When to use MCP-only vs MCP+AI

Use mcp_only when:

• You want deterministic/local command resolution.

• You prefer speed and strictness over open-ended reasoning.

• The request is explicit enough for planner/materializers.

Use mcp_ai when:

• Request is complex, ambiguous, or cross-domain.

• You need richer reasoning, explanation, or conflict tradeoffs.

• Deterministic planner reports unresolved intent and you want model help.