Nectar AI Facility Agent
Nectar AI Facility Agent
An autonomous AI facility operations assistant built for the Nectar Intelligent Facilities Platform challenge.
The system allows facility operators to interact naturally through voice and enables the agent to understand requests, route them to the appropriate workflow, retrieve facility knowledge using RAG, access live facility information through MCP tools, reason over multiple sources, safely perform operational actions, and respond through voice.
1. Problem Statement
Facility operators often need to investigate issues by checking multiple sources of information such as:
Building temperature
HVAC asset status
Sensor readings
Energy consumption
Active alerts
Equipment relationships
Maintenance procedures
Troubleshooting documentation
A conventional chatbot can answer questions but cannot reliably investigate operational problems using live facility data and internal documentation.
This project addresses that problem by combining:
Voice AI + LLM Routing + RAG + MCP + Agentic Reasoning + Tool Calling + Controlled Actions + Text-to-Speech
The goal is to provide an autonomous facility operations assistant rather than a simple question-answer chatbot.
2. Objectives
The system is designed to:
Receive natural language or voice input.
Convert speech to text.
Understand user intent.
Route the request to the appropriate agent workflow.
Retrieve information from facility documentation.
Query live facility data through MCP tools.
Combine multiple sources of information.
Reason about facility conditions.
Safely execute operational actions after confirmation.
Convert the final response back to speech.
Maintain conversational interaction.
Provide grounded responses and avoid unsupported claims.
3. High-Level Architecture
USER
|
Voice / Text
|
v
Speech-to-Text
|
v
FastAPI API
|
v
Agent / LLM Router
|
+---------------+----------------+
| | |
v v v
RAG MCP General
Agent Tools LLM
| |
v v
Facility Docs Live Facility
Knowledge Base Data
| |
+-------+-------+
|
v
Reasoning Layer
|
v
Decision / Response
|
+-------+--------+
| |
v v
MCP Action Answer
| |
v v
Maintenance Request Text-to-Speech
|
v
Voice Response
Technology Stack
Backend
Python
FastAPI
Uvicorn
Agent Orchestration
LangGraph
LLM-based routing
Agentic workflow
LLM
Google Gemini
RAG
LangChain
ChromaDB
HuggingFace Embeddings
Semantic retrieval
MCP
Model Context Protocol
MCP Server
MCP Client
Facility operation tools
Voice
SpeechRecognition
Browser Speech Recognition / Speech-to-Text
Browser Text-to-Speech
Frontend
HTML
CSS
JavaScript
Testing
Pytest
Conclusion
The Nectar AI Facility Agent demonstrates an autonomous AI workflow for intelligent facility operations.
The system combines:
Speech-to-Text
Text-to-Speech
LLM reasoning
Intelligent routing
RAG
Vector search
MCP
Tool calling
Synthetic facility data
Multi-step reasoning
Controlled operational actions
Confirmation-based safety
Automated testing
Conclusion
The Nectar AI Facility Agent demonstrates an autonomous AI workflow for intelligent facility operations.
The system combines:
Speech-to-Text
Text-to-Speech
LLM reasoning
Intelligent routing
RAG
Vector search
MCP
Tool calling
Synthetic facility data
Multi-step reasoning
Controlled operational actions
Confirmation-based safety
Automated testing
Conclusion
The Nectar AI Facility Agent demonstrates an autonomous AI workflow for intelligent facility operations.
The system combines:
Speech-to-Text
Text-to-Speech
LLM reasoning
Intelligent routing
RAG
Vector search
MCP
Tool calling
Synthetic facility data
Multi-step reasoning
Controlled operational actions
Confirmation-based safety
Automated testing
The key objective is to demonstrate how an AI agent can move beyond simple question answering and autonomously investigate facility problems using both organizational knowledge and live operational data while maintaining safety around operational actions.