Skip to main content
Glama

embodied-nav-mcp

방 안을 둘러보고, 이름을 부른 대상을 찾아가고, 보이는 것을 세고, 작업이 끝나면 멈출 수 있는 언어 모델입니다.

모델은 무엇을 할지 결정합니다. 센서는 사물이 어디에 있는지 결정합니다.

도구

도구

기능

get_view

360° 카메라의 4개 원근 뷰와 포즈

ground

지시 표현 → 픽셀 박스 → 라이다 미터 → 컨버터에 스냅된 웨이포인트

drive_to

맵 프레임 (x, y) 하나를 게시하고 도착하거나 정착할 때까지 대기

stop

플래너가 마지막 웨이포인트를 추적하지 않도록 정차

parse_numerical

세기 질문을 대상(target) / 기준(anchor)으로 분할

count_in_view

이 시점에서만 계산; 각 인스턴스는 라이다로 리프트됨

commit_count

위치별로 조회를 클러스터링; 정수를 게시

publish_object

마지막 ground의 라이다 AABB; 3D 마커 게시

commit_count는 모델로부터 숫자를 받지 않습니다. 베개 4개의 두 뷰는 8이 아닌 4가 됩니다.

지오메트리는 Team Xiao Hei의 feat/xiao-hei-instruction-merge Type 3 스택입니다: 크기 게이트, 블라인드 콘 처리, 컨버터 정착 포즈, 종료 시 주차.

Related MCP server: Webots MCP

설치

uv sync --extra dev
export ANTHROPIC_API_KEY=...
uv run pytest

Python 3.11+. 실제 로봇/시뮬레이션에는 ROS Jazzy와 자율주행 스택이 필요합니다. 테스트는 가짜 로봇을 사용하므로 둘 다 필요하지 않습니다.

사용해 보기

# Dry run, no ROS
uv run embodied-mcp-agent --fake "How many blue chairs are between the table and the wall?"
uv run embodied-mcp-agent --fake "Find the potted plant on the kitchen island closest to the fridge."
uv run embodied-mcp-agent --fake "Take the path near the window to the fridge."

스택이 실행 중인 로봇(또는 챌린지 컨테이너)에서:

uv run embodied-mcp-challenge
# evaluators still run: ros2 launch dummy_vlm dummy_vlm.launch

embodied-mcp --attach-ros는 stdio MCP를 통한 동일한 도구입니다. 챌린지 노드는 이를 프로세스 내에서 호출합니다: 로봇 컨테이너에는 원격 MCP 커넥터용 공개 URL이 없습니다.

CMU VLN 챌린지

드롭인 어댑터: examples/cmu_vln/. 패키지 이름 dummy_vlm은 변경되지 않았습니다.

유형

도구

ROS 출력

수치

parse_numerical, count_in_view, commit_count

/numerical_response

객체 참조

ground, publish_object

/selected_object_marker

지시 따르기

get_view, ground, drive_to, stop

/way_point_with_heading

허용 센서: /camera/image, /registered_scan, /terrain_map, /terrain_map_ext, /state_estimation. 도착은 /way_point_reached가 아닌 /state_estimation입니다. 프로세스당 질문 하나.

구성

변수

기본값

ANTHROPIC_API_KEY

—

필수

EMBODIED_MCP_MODEL

claude-opus-5

호스트 및 픽셀 접지 / 계산 모델

EMBODIED_MCP_BUDGET_S

540

프로세스 시작부터의 초

XIAO_HEI_IMAGE_TOPIC

/camera/image

카메라 토픽

라이선스

MIT. Team Xiao Hei의 벤더드 지오메트리 및 ROS I/O; NOTICE 참조.

Available Tools

8 tools
commit_countA

Cluster every count_in_view look by lidar position and publish one integer on /numerical_response. Two views of the same four pillows become 4, not 8. You do not pass the number — geometry does.

Inputs: no named parameters. Returns: result

count_in_viewA

Count matching instances from the current vantage point only. The call is never told the running total — do not add previous looks yourself. Each instance is boxed and lifted with lidar so later looks can be de-duplicated. If sufficient is false, drive as next_view suggests (or ground the anchor) and count_in_view again. When you are done looking, call commit_count — that publishes the clustered integer.

Inputs: target (required), question (required), anchor. Returns: result

drive_toA

Publish one waypoint (map-frame x, y) and wait until the vehicle arrives or settles. Arrival is decided from /state_estimation, not from a 'waypoint reached' topic. Prefer publish_xy from ground() over coordinates you invent.

Inputs: x (required), y (required), timeout_s. Returns: result

get_viewA

Capture the robot's current 360° camera, split into four perspective faces (front, right, back, left), and return them with the current pose. Call this when you need to see the room. Do not guess what is around you.

Inputs: no named parameters. Returns: result

groundA

Ground a referring expression in the live camera + lidar. Pass the object or place to go to, in the instruction's own words (e.g. 'the lantern closest to the fan decoration'). The tool asks a pixel-grounding model for a box, lifts metres from the registered scan, size-checks the range, and snaps the waypoint to where the platform converter will actually settle. Use publish_xy as the argument to drive_to. If committed is false, this is a step to a better viewpoint — drive there and ground again. If visible is false, drive to suggested_xy and look again.

Inputs: query (required). Returns: result

parse_numericalA

Split a counting question into target, attribute, relation, and anchor. Use this first on numerical questions so you know what to ground and what to pass to count_in_view.

Inputs: question (required). Returns: result

publish_objectA

Publish a 3D cube marker on /selected_object_marker for the object you last grounded. Call ground() on the unique referred object first. The box centre and size come from the lidar cluster, not from a guess.

Inputs: label (required). Returns: result

stopA

Park at the current pose so the local planner stops chasing the last waypoint. Call this when the instruction is finished, or after you have published a numerical or object-reference answer.

Inputs: no named parameters. Returns: result

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    A4.3/5.0

    Scored across 8 tools

    Disambiguation5/5

    Each tool serves a distinct function: perception (get_view), grounding (ground), navigation (drive_to, stop), numerical parsing (parse_numerical), counting (count_in_view), finalizing counts (commit_count), and object publication (publish_object). There is no overlap or ambiguous boundaries between them.

    Naming Consistency5/5

    All tool names follow a consistent verb_phrase pattern with snake_case (e.g., get_view, drive_to, commit_count). The verbs are imperative and descriptive, making the intent of each tool predictable from its name alone.

    Tool Count5/5

    With 8 tools, the server is well-scoped for embodied navigation and counting tasks. Each tool covers a necessary step in the workflow without redundancy or excessive granularity, fitting within the ideal 3-15 tool range.

    Completeness5/5

    The tool surface fully covers the lifecycle of the domain: perceiving the environment, grounding references, moving, stopping, parsing questions, counting from multiple views, committing aggregated results, and publishing object markers. There are no obvious gaps or dead ends for the intended tasks.

    Maintenance

    ActivityStale
    ResponsivenessNo recent community issues

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      A
      quality
      D
      maintenance
      Enables controlling robots in ROS environments through natural language, supporting topics, services, actions, and GUI tools.
      24
      36
      MIT
    • F
      license
      A
      quality
      D
      maintenance
      Enables agents to identify appliances and retrieve grounded, robot-executable operation packages, reducing hallucinations from 24% to nearly 0%.
      4
      -
    • F
      license
      Not graded
      quality
      D
      maintenance
      Enables LLM agents to control robot simulations (grasping, pouring) in the Genesis physics simulator through persistent sessions, allowing multi-turn interaction with the same scene.
      -