SAP AI Assistant --- MCP Gateway

Version: 1.0.0
Status: Initial MVP / Stable
Platform: SAP Business Technology Platform (BTP)
Runtime: Cloud Foundry / Node.js

1. Overview

SAP AI Assistant is an AI-powered interface for interacting with SAP BTP destinations and connected OData services using natural language. It combines SAP BTP Destination Service, Model Context Protocol (MCP), OData V2/V4, LangChain, Groq, Node.js, Cloud Foundry, and a browser-based chat UI.

The user selects a configured destination and asks questions such as Show me the first 5 products. The AI agent uses MCP tools to discover and query the selected OData service and returns a natural-language answer.

2. Architecture

                    SAP BTP
                       |
              +--------v--------+
              | Destination     |
              | Service         |
              +--------+--------+
                       |
              +--------v--------+
              | SAP AI Assistant|
              | MCP Gateway     |
              |                 |
              | Destination Mgmt|
              | OData Discovery |
              | OData Query     |
              | MCP Server      |
              | AI Agent        |
              +--------+--------+
                       |
              +--------v--------+
              |    Groq LLM     |
              | llama-3.3-70b   |
              +--------+--------+
                       |
                       v
                  Browser UI

3. Core Features

• SAP BTP Destination Service integration

• Dynamic destination selection

• Destination synchronization

• Destination lookup

• MCP server and MCP client

• OData metadata discovery

• OData V2 and V4 support

• Natural-language OData querying

• LangChain tool-calling agent

• Groq LLM integration

• User-friendly chat responses

• Cloud Foundry deployment

• Health endpoint

• Info and version dialogs

4. MCP Tools

Tool Purpose

list_destinations Lists synchronized destinations

get_destination Retrieves destination details

sync_destinations Synchronizes destinations from SAP BTP

discover_odata Discovers OData metadata and entity sets

query_odata Executes an OData query against a selected destination

5. Destination Flow

Destination names are not hard-coded into the AI agent. The client sends the selected destination with the question:

{
  "message": "Show me the first 5 products",
  "destination": "NORTHWIND_V4"
}

The backend passes both values to the agent:

UI -> /ai/chat -> askAgent({ question, destination }) -> MCP -> selected destination -> OData service

This allows destinations such as S4_PROD, S4_TEST, ARIBA, SUCCESSFACTORS, or customer-specific destinations without changing agent code.

6. HTTP Endpoints

Health

GET /health

Example response:

{ "status": "UP" }

List Destinations

GET /destinations

Destination Management

GET /destinationsom/destinations

Get Destination

GET /destination/:name

OData Discovery

GET /odata/:destination/discover

AI Chat

POST /ai/chat
Content-Type: application/json

Example request:

{
  "message": "Show me the first 5 products",
  "destination": "NORTHWIND_V4"
}

Example response:

{
  "success": true,
  "question": "Show me the first 5 products",
  "destination": "NORTHWIND_V4",
  "answer": "The first 5 products are..."
}

The UI displays the answer rather than exposing raw MCP/OData JSON to normal users.

7. AI Agent Flow

1. Receive the user question and selected destination.

2. Connect to the MCP server.

3. Retrieve available MCP tools.

4. Bind the tools to the LLM.

5. Allow the LLM to select the required tool(s).

6. Execute MCP tools.

7. Return tool results to the LLM.

8. Produce a natural-language answer.

Question
   |
   v
  LLM
   |
   +--> discover_odata
   |
   +--> query_odata
   |
   v
Tool Result
   |
   v
  LLM
   |
   v
Final Answer

8. Environment Variables

GROQ_API_KEY

Required by the AI agent. Configure it in Cloud Foundry rather than committing it to source control:

cf set-env sap-mcp-gateway GROQ_API_KEY "<your-groq-api-key>"
cf restage sap-mcp-gateway

MCP_SERVER_URL

Defines the MCP endpoint. Local default:

http://localhost:8080/mcp

For Cloud Foundry, configure the deployed MCP endpoint as appropriate.

9. Deployment

cf login -sso
cf target -o <organization> -s <space>
cf push sap-mcp-gateway
cf app sap-mcp-gateway
cf logs sap-mcp-gateway --recent

Configure the Groq key and restage:

cf set-env sap-mcp-gateway GROQ_API_KEY "<your-key>"
cf restage sap-mcp-gateway

10. Example Development Destinations

NORTHWIND_V2

Name:           NORTHWIND_V2
Type:           HTTP
URL:            https://services.odata.org/v2/northwind/northwind.svc/
Proxy Type:     Internet
Authentication: NoAuthentication

NORTHWIND_V4

Name:           NORTHWIND_V4
Type:           HTTP
URL:            https://services.odata.org/v4/northwind/northwind.svc/
Proxy Type:     Internet
Authentication: NoAuthentication

These are demonstration destinations. Production systems should use appropriate SAP/customer destinations and authentication.

11. Security

Version 1 focuses on the MCP and destination integration foundation. Before enterprise production use, review or implement:

• XSUAA authentication

• Role-based access control

• Destination-level authorization

• User identity propagation where required

• Secure secret handling

• Audit logging

• Rate limiting

• Input validation

• OData query restrictions

• Result limits

• Sensitive-data protection

• Production monitoring

Never commit API keys to Git.

12. Known v1 Limitations

AI token usage

The current agent can perform multiple LLM/tool iterations and may accumulate metadata and tool results in the conversation. Large metadata responses can therefore increase token consumption.

Agent routing

The current implementation lets the LLM decide which MCP tools to call. Future versions can add deterministic routing for common operations to reduce unnecessary calls.

Authentication

Enterprise authentication and authorization are planned for a subsequent phase.

13. Version 1.0 Scope

Included in v1.0:

• Cloud Foundry deployment

• Health endpoint

• Destination Service integration

• Destination synchronization

• Dynamic destination selection

• MCP server

• MCP client

• MCP tool discovery

• Destination tools

• OData discovery

• OData V2 support

• OData V4 support

• OData querying

• Groq integration

• LangChain agent

• AI chat endpoint

• Natural-language UI

• User-friendly response rendering

• Info dialog

• Version dialog

• Initial technical documentation

14. Roadmap

v1.1 --- AI Optimization

• Reduce unnecessary LLM iterations

• Reduce metadata sent to the LLM

• Limit query results

• Optimize tool-result handling

• Improve error recovery and latency

• Add deterministic routing where appropriate

v1.2 --- Security

• XSUAA

• User authentication

• RBAC

• Destination authorization

• Identity propagation

• Audit logging

v1.3 --- Enterprise OData

• $select

• $filter

• $orderby

• $expand

• $top

• $skip

• Pagination

• Result summarization

v2.0 --- SAP Enterprise MCP Platform

Potential integrations include S/4HANA, SuccessFactors, Ariba, SAP HANA Cloud, Integration Suite, SAP Build Process Automation, and SAP AI capabilities.

15. Architectural Principles

Destination agnostic

Customer-specific destination names should not be embedded in application logic.

Tool driven

The AI agent interacts with SAP systems through MCP tools.

Configuration over code

Environment-specific destinations and credentials remain outside application source code.

User-friendly output

Raw API and MCP JSON should not be exposed to normal users.

Extensible

New MCP tools should be addable without redesigning the entire application.

16. Release Information

Project: SAP AI Assistant / MCP Gateway
Version: 1.0.0
Status: Initial MVP / Stable
Platform: SAP BTP
Runtime: Node.js / Cloud Foundry