<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<!-- Meta tags for social media banners -->
<meta name="description" content="SensorMCP Server - Automated dataset creation and custom object detection model training through natural language interactions using the Model Context Protocol">
<meta property="og:title" content="SensorMCP Server - AI-Powered Computer Vision"/>
<meta property="og:description" content="Automated dataset creation and custom object detection model training through natural language interactions using the Model Context Protocol"/>
<meta property="og:url" content=""/>
<meta name="twitter:title" content="SensorMCP Server - AI-Powered Computer Vision">
<meta name="twitter:description" content="Automated dataset creation and custom object detection model training through natural language interactions">
<meta name="twitter:card" content="summary">
<!-- Keywords for your paper to be indexed by-->
<meta name="keywords" content="computer vision, object detection, MCP, model context protocol, autodistill, YOLOv8, GroundedSAM, automated labeling, dataset creation">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>SensorMCP Server</title>
<link rel="icon" type="image/x-icon" href="static/images/favicon.ico">
<link href="https://fonts.googleapis.com/css?family=Google+Sans|Noto+Sans|Castoro"
rel="stylesheet">
<link rel="stylesheet" href="static/css/bulma.min.css">
<link rel="stylesheet" href="static/css/fontawesome.all.min.css">
<link rel="stylesheet"
href="https://cdn.jsdelivr.net/gh/jpswalsh/academicons@1/css/academicons.min.css">
<link rel="stylesheet" href="static/css/index.css">
<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.5.1/jquery.min.js"></script>
<script defer src="static/js/fontawesome.all.min.js"></script>
<script src="static/js/index.js"></script>
</head>
<body>
<section class="hero">
<div class="hero-body">
<div class="container is-max-desktop">
<div class="columns is-centered">
<div class="column has-text-centered">
<h1 class="title is-1 publication-title">SensorMCP Server</h1>
<div class="is-size-5 publication-authors">
<!-- Project contributors -->
<span class="author-block">
<a href="mailto:yq@anysign.net" target="_blank">SensorMCP Team</a></span>
</div>
<div class="is-size-5 publication-authors">
<span class="author-block">Yunqi Guo, Guanyu Zhu, Kaiwei Liu, Guoliang Xing<br>2025</span>
</div>
<div class="column has-text-centered">
<div class="publication-links">
<!-- Documentation link -->
<span class="link-block">
<a href="README.md" target="_blank"
class="external-link button is-normal is-rounded is-dark">
<span class="icon">
<i class="fas fa-file-alt"></i>
</span>
<span>Documentation</span>
</a>
</span>
<!-- Github link -->
<span class="link-block">
<a href="https://github.com/sensormcp/sensor-mcp" target="_blank"
class="external-link button is-normal is-rounded is-dark">
<span class="icon">
<i class="fab fa-github"></i>
</span>
<span>Code</span>
</a>
</span>
<!-- MCP Protocol Link -->
<span class="link-block">
<a href="https://modelcontextprotocol.io/" target="_blank"
class="external-link button is-normal is-rounded is-dark">
<span class="icon">
<i class="fas fa-link"></i>
</span>
<span>MCP Protocol</span>
</a>
</span>
<!-- Demo Link -->
<!-- <span class="link-block">
<a href="#demo"
class="external-link button is-normal is-rounded is-primary">
<span class="icon">
<i class="fas fa-play"></i>
</span>
<span>Live Demo</span>
</a> -->
</span>
</div>
</div>
</div>
</div>
</div>
</div>
</section>
<!-- Introduction -->
<section class="hero teaser">
<div class="container is-max-desktop">
<div class="hero-body">
<h2 class="subtitle has-text-centered">
SensorMCP Server enables seamless integration between Large Language Models and computer vision workflows,
automatically creating labeled datasets and training custom object detection models through natural language interactions.
</h2>
</div>
</div>
</section>
<!-- End introduction -->
<!-- Paper abstract -->
<section class="section hero is-light">
<div class="container is-max-desktop">
<div class="columns is-centered has-text-centered">
<div class="column is-four-fifths">
<h2 class="title is-3">Overview</h2>
<div class="content has-text-justified">
<p>
<strong>SensorMCP Server</strong> combines the power of foundation models (like GroundedSAM) with custom model training (YOLOv8)
to create a seamless workflow for object detection. Using the Model Context Protocol, it enables Large Language Models to
automatically label images using foundation models, create custom object detection datasets, train specialized detection models,
and download images from Unsplash for training data.
</p>
<p>
The system supports a complete end-to-end workflow: from defining object ontologies through natural language,
to automatically labeling training data with foundation models, to fine-tuning custom YOLOv8 detection models.
All functionality is exposed through the Model Context Protocol, making it seamlessly accessible to LLM-powered applications and chat interfaces.
</p>
</div>
</div>
</div>
</div>
</section>
<!-- End paper abstract -->
<!-- Key Features -->
<section class="hero is-small">
<div class="hero-body">
<div class="container">
<h2 class="title is-3">Key Features</h2>
<div class="columns is-multiline">
<div class="column is-half">
<div class="content has-text-centered">
<h4><strong>Foundation Model Integration</strong></h4>
<p>Uses GroundedSAM for automatic image labeling and object detection.</p>
</div>
</div>
<div class="column is-half">
<div class="content has-text-centered">
<h4><strong>Custom Model Training</strong></h4>
<p>Fine-tune YOLOv8 models on your specific objects and datasets.</p>
</div>
</div>
<div class="column is-half">
<div class="content has-text-centered">
<h4><strong>MCP Protocol Integration</strong></h4>
<p>Native integration with LLM workflows and chat interfaces.</p>
</div>
</div>
<div class="column is-half">
<div class="content has-text-centered">
<h4><strong>Automated Dataset Creation</strong></h4>
<p>Download images from Unsplash and automatically create labeled datasets.</p>
</div>
</div>
</div>
</div>
</div>
</section>
<!-- End key features -->
<!-- Architecture Overview -->
<section class="hero is-small is-light">
<div class="hero-body">
<div class="container">
<h2 class="title is-3">Architecture & Workflow</h2>
<div class="columns is-centered has-text-centered">
<div class="column is-four-fifths">
<div class="content has-text-justified">
<h4><strong>1. Ontology Definition</strong></h4>
<p>Define object classes to detect through natural language (e.g., "tiger, elephant, zebra")</p>
<h4><strong>2. Foundation Model Setup</strong></h4>
<p>Initialize GroundedSAM as the base model for automatic image labeling</p>
<h4><strong>3. Data Acquisition</strong></h4>
<p>Download images from Unsplash or import local image collections</p>
<h4><strong>4. Automatic Labeling</strong></h4>
<p>Use foundation models to automatically generate labels and bounding boxes</p>
<h4><strong>5. Model Training</strong></h4>
<p>Fine-tune YOLOv8 models on the automatically labeled dataset</p>
<h4><strong>6. MCP Integration</strong></h4>
<p>All functionality exposed through Model Context Protocol for LLM integration</p>
</div>
</div>
</div>
</div>
</div>
</section>
<!-- Demo Workflow -->
<section class="hero is-small">
<div class="hero-body">
<div class="container">
<h2 class="title is-3">Demo Workflow</h2>
<div class="columns is-multiline">
<div class="column is-one-third">
<div class="content has-text-centered">
<h4><strong>1. Setting up Ontology</strong></h4>
<p>Defining object classes through natural language commands.</p>
</div>
</div>
<div class="column is-one-third">
<div class="content has-text-centered">
<h4><strong>2. Automatic Labeling</strong></h4>
<p>Foundation models automatically generating labels and bounding boxes.</p>
</div>
</div>
<div class="column is-one-third">
<div class="content has-text-centered">
<h4><strong>3. Model Training</strong></h4>
<p>Training custom YOLOv8 models on automatically labeled datasets.</p>
</div>
</div>
</div>
</div>
</div>
</section>
<!-- End demo workflow -->
<!-- Technical Specifications -->
<section class="section hero is-light" id="demo">
<div class="container is-max-desktop">
<div class="columns is-centered has-text-centered">
<div class="column is-four-fifths">
<h2 class="title is-3">Technical Specifications</h2>
<div class="content has-text-left">
<div class="columns">
<div class="column">
<h4><strong>Supported Base Models</strong></h4>
<ul>
<li>GroundedSAM - Foundation model for object detection and segmentation</li>
</ul>
<h4><strong>Supported Target Models</strong></h4>
<ul>
<li>YOLOv8n.pt - Nano (fastest inference)</li>
<li>YOLOv8s.pt - Small (balanced speed/accuracy)</li>
<li>YOLOv8m.pt - Medium (higher accuracy)</li>
<li>YOLOv8l.pt - Large (high accuracy)</li>
<li>YOLOv8x.pt - Extra Large (highest accuracy)</li>
</ul>
</div>
<div class="column">
<h4><strong>MCP Tools Available</strong></h4>
<ul>
<li><code>list_available_models()</code></li>
<li><code>define_ontology(objects_list)</code></li>
<li><code>set_base_model(model_name)</code></li>
<li><code>set_target_model(model_name)</code></li>
<li><code>fetch_unsplash_images(query, max_images)</code></li>
<li><code>import_images_from_folder(folder_path)</code></li>
<li><code>label_images()</code></li>
<li><code>train_model(epochs, device)</code></li>
</ul>
</div>
</div>
</div>
</div>
</div>
</div>
</section>
<!-- Installation Guide -->
<section class="hero is-small is-light">
<div class="hero-body">
<div class="container">
<h2 class="title">Quick Start</h2>
<div class="content">
<h4><strong>Installation</strong></h4>
<pre><code>git clone <repository-url>
cd sensor-mcp
uv sync</code></pre>
<h4><strong>MCP Configuration</strong></h4>
<pre><code>{
"mcpServers": {
"autodistill-server": {
"type": "stdio",
"command": "uv",
"args": [
"--directory",
"/path/to/sensor-mcp",
"run",
"src/zoo_mcp.py"
]
}
}
}</code></pre>
<h4><strong>Environment Setup</strong></h4>
<pre><code>UNSPLASH_API_KEY=your_unsplash_api_key_here</code></pre>
</div>
</div>
</div>
</section>
<!-- Citation -->
<section class="section" id="Citation">
<div class="container is-max-desktop content">
<h2 class="title">Citation</h2>
<p>If you use this code or data in your research, please cite our paper:</p>
<pre><code>@inproceedings{Guo2025,
author = {Guo, Yunqi and Zhu, Guanyu and Liu, Kaiwei and Xing, Guoliang},
title = {A Model Context Protocol Server for Custom Sensor Tool Creation},
booktitle = {3rd International Workshop on Networked AI Systems (NetAISys '25)},
year = {2025},
month = {jun},
address = {Anaheim, CA, USA},
publisher = {ACM},
doi = {10.1145/3711875.3736687},
isbn = {979-8-4007-1453-5/25/06}
}</code></pre>
</div>
</section>
<!--Contact -->
<section class="section" id="Contact">
<div class="container is-max-desktop content">
<h2 class="title">Contact</h2>
<p>For questions about the zoo dataset or technical inquiries:</p>
<p><strong>Email:</strong> <a href="mailto:yq@anysign.net">yq@anysign.net</a></p>
<p>For MCP protocol documentation: <a href="https://modelcontextprotocol.io/" target="_blank">Model Context Protocol</a></p>
</div>
</section>
<footer class="footer">
<div class="container">
<div class="columns is-centered">
<div class="column is-8">
<div class="content">
<p>
This page was built using the <a href="https://github.com/eliahuhorwitz/Academic-project-page-template" target="_blank">Academic Project Page Template</a> which was adopted from the <a href="https://nerfies.github.io" target="_blank">Nerfies</a> project page.
You are free to borrow the source code of this website, we just ask that you link back to this page in the footer. <br> This website is licensed under a <a rel="license" href="http://creativecommons.org/licenses/by-sa/4.0/" target="_blank">Creative
Commons Attribution-ShareAlike 4.0 International License</a>.
</p>
</div>
</div>
</div>
</div>
</footer>
</body>
</html>
