nrs_llm

nrs_llm is an MCP server for ROS 2 environments. It connects to ROS through rosbridge WebSocket and exposes tools for:

• ROS 2 topic / service introspection

• topic publish / subscribe workflows

• service calls

• robot connectivity checks

• polishing log filtering and analysis

• polishing path regeneration

• TXT-based polishing path execution and logging

This repository is designed around the following environment assumptions:

• Ubuntu 22.04

• ROS 2 Humble

• Python 3.10+

• a separate ROS 2 workspace at ~/ros2_ws

• rosbridge_server running on 127.0.0.1:9090

This README is written so that a user can set up and run nrs_llm from scratch.

1. Recommended directory layout

The current codebase is easiest to run when you keep the following layout:

~/nrs_llm
~/ros2_ws
├── src
│   └── ... your ROS 2 packages
├── build
├── install
├── log
└── data
    └── ep
        ├── xyz.txt
        ├── vxyz.txt
        ├── fxyz.txt
        ├── rpy.txt
        ├── real_flat_filtered.txt
        ├── auto_logger.py
        ├── filter_logs_txt.py
        ├── polish_pipeline.py
        └── ...

If you move these paths, some parts of the polishing pipeline may fail unless you also update the corresponding paths in the code.

Related MCP server: Rosbridge MCP Server

2. System requirements

Install the base system tools first.

2.1 Ubuntu packages

sudo apt update
sudo apt install -y \
    python3 \
    python3-pip \
    python3-venv \
    python3-colcon-common-extensions \
    git

2.2 ROS 2 Humble

This README assumes that ROS 2 Humble is already installed at:

/opt/ros/humble

Check it with:

test -f /opt/ros/humble/setup.bash && echo "ROS 2 Humble found" || echo "ROS 2 Humble missing"

If this prints ROS 2 Humble missing, install ROS 2 Humble first before continuing.

3. Clone the repository

Clone this repository into your home directory:

cd ~
git clone <YOUR_REPOSITORY_URL> nrs_llm
cd ~/nrs_llm

If the repository is already cloned, just move into it:

cd ~/nrs_llm

4. Create and activate the Python virtual environment

This project should be run inside a dedicated virtual environment.

Create the environment:

cd ~/nrs_llm
python3 -m venv env_llm

Activate it:

source ~/nrs_llm/env_llm/bin/activate

After activation, your terminal prompt should show something like:

(env_llm) user@host:~/nrs_llm$

Upgrade pip:

python -m pip install --upgrade pip

To deactivate later:

deactivate

To activate again in a new terminal:

cd ~/nrs_llm
source env_llm/bin/activate

5. Install Python dependencies

Install the required Python packages inside env_llm.

5.1 Core packages

cd ~/nrs_llm
source env_llm/bin/activate

pip install \
    fastmcp \
    pillow \
    numpy \
    scipy \
    matplotlib \
    pandas \
    opencv-python \
    websocket-client

5.2 Why these packages are needed

• fastmcp: MCP server framework

• pillow: image handling

• numpy, scipy, matplotlib, pandas: filtering / analysis utilities

• opencv-python: required because utils/websocket_manager.py imports cv2

• websocket-client: required because utils/websocket_manager.py imports websocket

5.3 Verify imports

Run this check after installation:

cd ~/nrs_llm
source env_llm/bin/activate
python - <<'PY'
import cv2
import websocket
import numpy
import pandas
import scipy
import matplotlib
from PIL import Image
print("All required Python imports succeeded.")
PY

If this command succeeds, the Python environment is ready.

6. Prepare the ROS 2 workspace

This project assumes a ROS 2 workspace at ~/ros2_ws.

Create it if needed:

mkdir -p ~/ros2_ws/src

Place your required ROS 2 packages into ~/ros2_ws/src.

The current version assumes that the workspace includes the packages needed for your robot setup and polishing execution pipeline. In the previous version, ur_custom_ik and the executable txt_ik_executor_force_patched were used by the path execution flow.

7. Build the ROS 2 workspace

Build the workspace:

cd ~/ros2_ws
colcon build

Source the environment:

source /opt/ros/humble/setup.bash
source ~/ros2_ws/install/setup.bash

To confirm that the workspace source file exists:

test -f ~/ros2_ws/install/setup.bash && echo "ROS workspace found" || echo "ROS workspace missing"

If the file is missing, the workspace has not been built successfully yet.

8. Install and run rosbridge

nrs_llm does not talk directly to ROS 2. It communicates through rosbridge WebSocket.

8.1 Install rosbridge if needed

If rosbridge_server is not installed yet:

sudo apt update
sudo apt install -y ros-humble-rosbridge-server

8.2 Run rosbridge

Open a new terminal and run:

source /opt/ros/humble/setup.bash
source ~/ros2_ws/install/setup.bash
ros2 launch rosbridge_server rosbridge_websocket_launch.xml

By default, rosbridge listens on:

127.0.0.1:9090

8.3 Verify rosbridge is listening

In another terminal:

ss -ltnp | grep 9090

If rosbridge is running correctly, you should see a listener on port 9090.

9. Verify the repository structure

Before connecting nrs_llm to Gemini CLI, confirm that these files exist:

test -f ~/nrs_llm/server.py && echo "server.py found" || echo "server.py missing"
test -f ~/ros2_ws/install/setup.bash && echo "workspace setup found" || echo "workspace setup missing"
test -f /opt/ros/humble/setup.bash && echo "ROS Humble setup found" || echo "ROS Humble setup missing"
test -x ~/nrs_llm/env_llm/bin/python && echo "venv python found" || echo "venv python missing"

All four checks should succeed.

10. Configure Gemini CLI MCP in settings.json

nrs_llm is intended to be launched by Gemini CLI as an MCP server using stdio.

That means:

• do not manually keep python server.py running in another terminal

• Gemini CLI should launch the server itself

• the config should source both ROS environments before starting server.py

10.1 Edit the Gemini CLI settings.json

Open the same settings.json file that Gemini CLI is already reading for MCP configuration and add the following server entry:

{
  "mcpServers": {
    "ros-mcp": {
      "command": "/bin/bash",
      "args": [
        "-lc",
        "source /opt/ros/humble/setup.bash && source /home/eunseop/ros2_ws/install/setup.bash && cd /home/eunseop/nrs_llm && exec /home/eunseop/nrs_llm/env_llm/bin/python /home/eunseop/nrs_llm/server.py"
      ]
    }
  }
}

10.2 Important notes about this config

Replace paths if your username or directories are different.

For the current setup shown above:

• ROS 2 Humble source file: /opt/ros/humble/setup.bash

• ROS workspace source file: /home/eunseop/ros2_ws/install/setup.bash

• repository path: /home/eunseop/nrs_llm

• virtual environment Python: /home/eunseop/nrs_llm/env_llm/bin/python

10.3 Why /bin/bash -lc is used

This is important because it allows Gemini CLI to:

1. source ROS 2 Humble

2. source your ROS 2 workspace

3. change directory into the repository

4. run the correct virtual environment Python

Without this, the server may fail with missing ROS packages, missing Python modules, or broken relative paths.

11. Start Gemini CLI and verify the MCP server

Once settings.json is updated, start Gemini CLI from inside the repository environment:

cd ~/nrs_llm
source env_llm/bin/activate
gemini -y

Inside Gemini CLI, run:

/mcp list

If everything is correct, you should see ros-mcp in the list with a status similar to:

Ready

If the server is connected correctly, Gemini CLI should also show the list of available tools.

12. First recommended checks after connection

Once /mcp list shows ros-mcp as ready, test the server in this order:

1. connect_to_robot

2. get_topics

3. get_services

4. get_topic_type

5. get_service_type

6. publish_once or a read-only introspection tool

This helps distinguish between:

• MCP startup issues

• rosbridge connectivity issues

• ROS graph visibility issues

• robot-specific execution issues

13. Running the server manually for debugging

Even though Gemini CLI should launch the MCP server in normal use, manual execution is still useful for debugging.

13.1 Manual debug run

cd ~/nrs_llm
source env_llm/bin/activate
source /opt/ros/humble/setup.bash
source ~/ros2_ws/install/setup.bash
python ~/nrs_llm/server.py

13.2 Better debug check with stdout / stderr separation

timeout 5s /bin/bash -lc 'source /opt/ros/humble/setup.bash && source /home/eunseop/ros2_ws/install/setup.bash && cd /home/eunseop/nrs_llm && /home/eunseop/nrs_llm/env_llm/bin/python /home/eunseop/nrs_llm/server.py' >/tmp/ros_mcp_stdout.log 2>/tmp/ros_mcp_stderr.log

echo "===== STDOUT ====="
sed -n '1,80p' /tmp/ros_mcp_stdout.log
echo "===== STDERR ====="
sed -n '1,80p' /tmp/ros_mcp_stderr.log

This is useful for catching import errors such as missing cv2 or missing websocket support.

14. Default paths used by the polishing pipeline

The current codebase assumes the following default paths for filtering / analysis / regeneration / execution:

Filtering input

• ~/ros2_ws/data/ep/xyz.txt

• ~/ros2_ws/data/ep/vxyz.txt

• ~/ros2_ws/data/ep/fxyz.txt

• ~/ros2_ws/data/ep/rpy.txt

Analysis input/output

• ~/ros2_ws/data/ep/vxyz_filtered.txt

• ~/ros2_ws/data/ep/fxyz_filtered.txt

• output directory: ~/ros2_ws/data/ep/polish_out_filtered

Regeneration

• input path: ~/ros2_ws/data/ep/real_flat_filtered.txt

• removal map: ~/ros2_ws/data/ep/polish_out_filtered/removal_map.npz

• output path: ~/ros2_ws/data/ep/real_flat_filtered_new2.txt

Path execution

• logger: ~/ros2_ws/data/ep/auto_logger.py

• executor: ros2 run ur_custom_ik txt_ik_executor_force_patched

If any of these files or directories are missing, the corresponding pipeline stage may fail.

15. Common errors and fixes

15.1 ModuleNotFoundError: No module named 'cv2'

Cause:

• opencv-python is not installed in env_llm

Fix:

cd ~/nrs_llm
source env_llm/bin/activate
pip install opencv-python

15.2 ModuleNotFoundError: No module named 'websocket'

Cause:

• websocket-client is not installed in env_llm

Fix:

cd ~/nrs_llm
source env_llm/bin/activate
pip install websocket-client

Note: the import name is websocket, but the package you install is websocket-client.

15.3 /mcp list shows Disconnected

Common causes:

• Gemini CLI is not reading the settings.json file you edited

• the MCP server command is wrong

• the ROS environment is not sourced

• the virtual environment Python path is wrong

• server.py crashes at startup because of missing dependencies

• rosbridge is not running

Checklist:

test -x /home/eunseop/nrs_llm/env_llm/bin/python && echo OK
test -f /home/eunseop/nrs_llm/server.py && echo OK
test -f /opt/ros/humble/setup.bash && echo OK
test -f /home/eunseop/ros2_ws/install/setup.bash && echo OK

Then verify rosbridge:

ss -ltnp | grep 9090

Then run the 5-second debug command from Section 13.2.

15.4 No executable found

Cause:

• required ROS package was not built

• workspace was not sourced

• executable name is different from what the code expects

Fix:

cd ~/ros2_ws
colcon build
source /opt/ros/humble/setup.bash
source ~/ros2_ws/install/setup.bash
ros2 pkg executables ur_custom_ik

15.5 No such file or directory

Cause:

• missing files under ~/ros2_ws/data/ep

• missing polishing scripts

• wrong path assumptions inside the code

Fix:

ls ~/ros2_ws/data/ep

Check that all required TXT files and scripts actually exist.

15.6 pip warns about unrelated packages such as nrs-imitation

If your virtual environment accidentally contains unrelated editable installs, pip may show dependency warnings that are not directly related to nrs_llm.

To check:

pip show nrs-imitation

If you do not want that package inside env_llm, remove it:

pip uninstall -y nrs-imitation

16. Full startup sequence (recommended)

Use the following sequence every time.

Terminal 1: rosbridge

source /opt/ros/humble/setup.bash
source ~/ros2_ws/install/setup.bash
ros2 launch rosbridge_server rosbridge_websocket_launch.xml

Terminal 2: Gemini CLI

cd ~/nrs_llm
source env_llm/bin/activate
gemini -y

Inside Gemini CLI:

/mcp list

Expected result:

• ros-mcp appears

• status is Ready

• the MCP tools are listed

17. Quick command summary

One-time setup

cd ~
git clone <YOUR_REPOSITORY_URL> nrs_llm
cd ~/nrs_llm
python3 -m venv env_llm
source env_llm/bin/activate
python -m pip install --upgrade pip
pip install fastmcp pillow numpy scipy matplotlib pandas opencv-python websocket-client

ROS workspace build

mkdir -p ~/ros2_ws/src
cd ~/ros2_ws
colcon build

rosbridge run

source /opt/ros/humble/setup.bash
source ~/ros2_ws/install/setup.bash
ros2 launch rosbridge_server rosbridge_websocket_launch.xml

Gemini CLI run

cd ~/nrs_llm
source env_llm/bin/activate
gemini -y

18. Summary

To run nrs_llm successfully, you need all of the following:

1. ROS 2 Humble installed

2. a built ROS 2 workspace at ~/ros2_ws

3. rosbridge running on port 9090

4. the repository cloned at ~/nrs_llm

5. the env_llm virtual environment created and activated

6. all required Python dependencies installed

7. Gemini CLI settings.json configured to launch server.py through /bin/bash -lc

If you follow this README exactly, you should be able to launch Gemini CLI, run /mcp list, and see ros-mcp in the Ready state.