@arizeai/phoenix-mcp

{ "cells": [ { "cell_type": "markdown", "metadata": { "id": "Dl-e7qmlmAtu" }, "source": [ "<center>\n", " <p style=\"text-align:center\">\n", " <img alt=\"phoenix logo\" src=\"https://storage.googleapis.com/arize-phoenix-assets/assets/phoenix-logo-light.svg\" width=\"200\"/>\n", " <br>\n", " <a href=\"https://arize.com/docs/phoenix/\">Docs</a>\n", " |\n", " <a href=\"https://github.com/Arize-ai/phoenix\">GitHub</a>\n", " |\n", " <a href=\"https://arize-ai.slack.com/join/shared_invite/zt-2w57bhem8-hq24MB6u7yE_ZF_ilOYSBw#/shared-invite/email\">Community</a>\n", " </p>\n", "</center>\n", "<h1 align=\"center\">LLM Ops - Manual Instrumentation</h1>\n", "\n", "In this tutorial we will learn how to manually instrument an LLM powered application.\n", "\n", "The application processes user input text, utilizing a language model to determine whether the intent behind the input is to make a purchase or to seek information.\n", "* If the intent is identified as a purchase, the application recommends the most suitable product matching the user's preferences.\n", "* Conversely, if the user's intent is to inquire, the application retrieves the most relevant document to provide an accurate answer to the question posed.\n", "\n", "\n", "⚠️ This tutorial requires an OpenAI key to run\n", "\n", "" ] }, { "cell_type": "markdown", "metadata": { "id": "7aNKdhI5vDJN" }, "source": [ "# Install Libraries" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "!pip install openai \\\n", " opentelemetry-api \\\n", " opentelemetry-sdk \\\n", " openinference-semantic-conventions \\\n", " openinference-instrumentation-openai \\\n", " opentelemetry-exporter-otlp \\\n", " arize-phoenix \\\n", " 'httpx<0.28'" ] }, { "cell_type": "markdown", "metadata": { "id": "EGbQrmc1vid6" }, "source": [ "## Import Libraries" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import json\n", "import os\n", "from getpass import getpass\n", "from io import StringIO\n", "\n", "import openai\n", "import opentelemetry\n", "import pandas as pd\n", "from openai import OpenAI\n", "from openinference.instrumentation.openai import OpenAIInstrumentor\n", "from openinference.semconv.trace import OpenInferenceSpanKindValues, SpanAttributes\n", "from opentelemetry import trace as trace_api\n", "from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter\n", "from opentelemetry.sdk import trace as trace_sdk\n", "from opentelemetry.sdk.resources import Resource\n", "from opentelemetry.sdk.trace.export import SimpleSpanProcessor" ] }, { "cell_type": "markdown", "metadata": { "id": "EXVOHnY8DChH" }, "source": [ "## Custom Tracing\n", "\n", "Create the tracer provider setup to Phoenix." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "resource = Resource(attributes={})\n", "tracer_provider = trace_sdk.TracerProvider(resource=resource)\n", "span_exporter = OTLPSpanExporter(endpoint=\"http://localhost:6006/v1/traces\")\n", "span_processor = SimpleSpanProcessor(span_exporter=span_exporter)\n", "tracer_provider.add_span_processor(span_processor=span_processor)\n", "trace_api.set_tracer_provider(tracer_provider=tracer_provider)\n", "\n", "tracer = trace_api.get_tracer(__name__)\n", "\n", "# Because we are using Open AI, we will use this along with our custom instrumentation\n", "OpenAIInstrumentor().instrument(skip_dep_check=True)" ] }, { "cell_type": "markdown", "metadata": { "id": "y7eykx3XwI_i" }, "source": [ "## Initialize Phoenix UI\n", "\n", "This will take a few seconds to start up the UI" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import phoenix as px\n", "\n", "px.launch_app().view()" ] }, { "cell_type": "markdown", "metadata": { "id": "6QgFJE1-38V5" }, "source": [ "### Create Sample Data\n", "\n", "Sample data is dispersed in three DataFrames:\n", "\n", "* `my_items` - Item data (Item, Price (USD),\tDescription, Stars, Best Use, Material, and Warranty)\n", "\n", "* `policy_data` - Policy questions, answers, and category\n", "\n", "* `customer_inputs` - Customer questions/ inquiries (Customer ID, Premium Customer, and question)\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "my_items = \"\"\"\n", "Item|Price (USD)|Description|Stars|Best Use|Material|Warranty\n", "Smart LED Light Bulb|$19.99|Control your lights remotely with your phone. Compatible with Alexa and Google Assistant.|4.5|Home automation|Plastic|1 year\n", "Portable Bluetooth Speaker|$39.99|Waterproof, with 10-hour battery life. Connects seamlessly with Bluetooth-enabled devices.|4.3|Outdoor activities|Aluminum|2 years\n", "Stainless Steel Water Bottle|$24.95|Double-walled, vacuum-insulated bottle. Keeps drinks hot for 12 hours and cold for 24 hours.|4.7|Gym and travel|Stainless steel|Lifetime\n", "Wireless Charging Pad|$29.99|Charges compatible smartphones without the need for cables. Sleek design with LED indicator.|4.2|Office and home|Plastic|1 year\n", "Fitness Tracker Watch|$49.95|Tracks steps, heart rate, and sleep quality. Syncs with smartphone for notifications.|4.4|Fitness and health|Silicone|2 years\n", "Resistance Bands Set|$14.99|Set of 5 bands with varying resistance levels. Perfect for home workouts or gym sessions.|4.6|Home and gym|Latex|1 year\n", "Electric Kettle|$34.99|Boils water quickly and efficiently. Auto-shutoff feature for safety. Capacity: 1.7 liters.|4.8|Kitchen|Stainless steel|2 years\n", "Travel Neck Pillow|$19.99|Memory foam pillow with adjustable closure. Provides support during long flights or road trips.|4.5|Travel|Memory foam|1 year\n", "Kindle E-reader|$89.99|High-resolution display with built-in adjustable light. Holds thousands of books. Battery lasts weeks.|4.9|Reading|Plastic|1 year\n", "Reusable Silicone Food Storage Bags|$12.95|Eco-friendly alternative to plastic bags. Dishwasher and microwave safe. Set of 6 in various sizes.|4.7|Kitchen|Silicone|2 years\n", "\"\"\"\n", "\n", "policy_data = \"\"\"\n", "Question|Answer|Category\n", "What is your return policy?|Our return policy lasts 30 days. If 30 days have gone by since your purchase, unfortunately, we can’t offer you a refund or exchange. To be eligible for a return, your item must be unused and in the same condition that you received it. It must also be in the original packaging.|Return Policy\n", "How long does delivery take?|Standard delivery times vary by location. Orders within the continental U.S. typically arrive within 3-5 business days. International deliveries can take anywhere from 7-21 business days, depending on customs and local delivery speeds.|Delivery Time\n", "Do you offer international shipping?|Yes, we ship to over 100 countries worldwide. Shipping costs and times vary depending on the destination. All applicable duties and taxes will be paid by the recipient.|International Shipping\n", "Can I change or cancel my order after placing it?|You can change or cancel your order within 1 hour of placing it. Please contact our customer service team as soon as possible. Once the order has moved to the processing stage, we're unable to cancel or make changes.|Order Modification\n", "What payment methods do you accept?|We accept all major credit cards, PayPal, and Apple Pay. For certain countries, we also accept local payment methods; these will be displayed at checkout.|Payment Options\n", "Is it safe to shop on your website?|Absolutely. We use SSL encryption to ensure all your personal information is encrypted before transmission. We do not store credit card details nor have access to your credit card information.|Security\n", "What do I do if my order arrives damaged or incorrect?|Please contact us within 48 hours of receiving your order with photographic evidence of the damage or incorrect item. We will arrange for a replacement or refund as quickly as possible.|Damaged or Incorrect Orders\n", "How can I track my order?|Once your order has been shipped, you will receive an email with a tracking number and a link to track your package.|Order Tracking\n", "Do you offer gift wrapping services?|Yes, we offer gift wrapping for a small additional charge. You can select this option at checkout and include a personalized message if desired.|Gift Services\n", "What is your policy on sustainability and eco-friendliness?|We're committed to reducing our environmental impact. We use eco-friendly packaging and partner with suppliers who prioritize sustainable practices. Additionally, we support various environmental initiatives each year.|Sustainability\n", "\"\"\"\n", "\n", "customer_inputs = \"\"\"\n", "Customer ID|Premium Customer|Customer Input\n", "Cust789|Yes|I need a new Kindle E-reader for my reading hobby. Are there any discounts currently?\n", "Cust456|No|Looking for a durable water bottle for my daily runs. Is the Stainless Steel Water Bottle available?\n", "Cust567|Yes|How can I track my Portable Bluetooth Speaker order?\n", "Cust123|Yes|I'm interested in smart home gadgets. Do you have the Smart LED Light Bulb in stock?\n", "Cust234|No|What is your return policy for the Kindle E-reader if I'm not satisfied?\n", "Cust890|No|Do you offer international shipping for the Fitness Tracker Watch?\n", "\"\"\"" ] }, { "cell_type": "markdown", "metadata": { "id": "5kYqAC0JflP2" }, "source": [ "### Format Customer Data" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "items_df = pd.read_csv(StringIO(my_items.strip()), delimiter=\"|\")\n", "policy_df = pd.read_csv(StringIO(policy_data.strip()), delimiter=\"|\")\n", "customer_inputs_df = pd.read_csv(StringIO(customer_inputs.strip()), delimiter=\"|\")" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "items_df.head()" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "policy_df.head()" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "customer_inputs_df.head()" ] }, { "cell_type": "markdown", "metadata": { "id": "i5Cvf5uLDPfQ" }, "source": [ "### Input OpenAI API Key\n", "\n", "When prompted, enter your API key." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "if not (openai_api_key := os.getenv(\"OPENAI_API_KEY\")):\n", " openai_api_key = getpass(\"🔑 Enter your OpenAI API key: \")\n", "openai.api_key = openai_api_key\n", "os.environ[\"OPENAI_API_KEY\"] = openai_api_key" ] }, { "cell_type": "markdown", "metadata": { "id": "TKshtt2ZF2lZ" }, "source": [ "### Create Prompts\n", "\n", "We'll create prompts for tracing data we wish to collect.\n", "\n", "\n", "* Customer intent prompt - Classifies if the customer is asking about a purchase or general inquiry\n", "* Customer QA Prompt - Prompt to help answer general inquiries\n", "* Item Search Prompt - Prompt about purchase questions" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "customer_intent_prompt = \"\"\"\n", "You are a helpful assistant designed to output JSON. Classify the following customer text as either a 'purchase' or 'query'.\n", "\n", "To help define the difference between purchae and query:\n", "- A purchase is a customer asking about a specific item or function of an item with the intent to purchase.\n", "- A query is likely asking about policies on returns, shipping, order modifications, and general inquiries outside of seeking to purchase an item.\n", "\n", "Choose the \"purchase\" category if you see both purchase and query intent.\n", "key: customer_intent\n", "value: 'purchase' or a 'query'\n", "\n", "If intent is purchase, append another key 'shopping_category' and the value should be one of the following:\n", "['Fitness and health',\n", " 'Gym and travel',\n", " 'Home and gym',\n", " 'Home automation',\n", " 'Kitchen',\n", " 'Office and home',\n", " 'Outdoor activities',\n", " 'Reading',\n", " 'Travel']\n", "\n", "If intent is query, append another key 'query_category' and the value should be one of the following:\n", "['Damaged or Incorrect Orders',\n", " 'Delivery Time',\n", " 'Gift Services',\n", " 'International Shipping',\n", " 'Order Modification',\n", " 'Order Tracking',\n", " 'Payment Options',\n", " 'Return Policy',\n", " 'Security',\n", " 'Sustainability']\n", "\"\"\"\n", "\n", "customer_qa_prompt = \"\"\"\n", "You are a helpful assistant designed to output JSON. Assist with answering customer queries about policies on returns, shipping, order modifications, and general inquiries for an e-commerce shop.\n", "\n", "When responding to a customer query, carefully consider the context of their question and provide a clear, detailed response. Your response should informatively guide the customer on the next steps they can take or the information they're seeking.\n", "\n", "Output JSON where the key is \"customer_response\" and the value is your objective and detailed answer to the customer's query. If additional policy details are relevant, include them in your response to ensure the customer receives complete and accurate guidance.\n", "\n", "Structure your response as follows:\n", "\n", "key: \"customer_response\"\n", "value: \"<Your objective, detailed response here>\"\n", "\n", "The objective is to fully address the customer's concern, providing them with precise information and clear next steps where applicable, without unnecessary embellishments.\n", "\"\"\"\n", "\n", "item_search_prompt = \"\"\"\n", "You are a helpful assistant designed to output JSON. Support the shopping process for customers in an e-commerce shop.\n", "\n", "When an item matches the customer's search criteria, your response should offer a concise and objective description of the item, focusing on its features, price, how it addresses their product search and any relevant details pertinent to the customer's needs.\n", "\n", "Output JSON where the key is \"customer_response\" and the value is a thorough description of the item. Highlight the item's features and specifications that meet the customer's requirements and any additional information necessary for an informed purchase.\n", "\n", "Structure your response as follows:\n", "\n", "key: \"customer_response\"\n", "value: \"<Your detailed, objective description here>\"\n", "\n", "The goal is to equip the customer with all the necessary information about the item, focusing on providing factual and relevant details to assist them in their decision-making process.\n", "\"\"\"" ] }, { "cell_type": "markdown", "metadata": { "id": "4y0aNEnRsjPr" }, "source": [ "## Define Functions\n", "\n", "Each function performs a specific task in manual instrumentation. Refer to function descriptions for more information." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "def openai_classify_user_intent(\n", " user_prompt: str, user_payload_json: str, tracer: opentelemetry.sdk.trace.Tracer\n", ") -> str:\n", " \"\"\"\n", " Classify the user intent as either a purchase or an inquiry using the OpenAI API\n", "\n", " Parameters\n", " ----------\n", " user_prompt : str\n", " Prompt template for OpenAI API\n", " user_payload_json : str\n", " User JSON payload with Customer ID, Customer Input, and Premium Customer\n", " tracer : opentelemetry.sdk.trace.Tracer\n", " Tracer to handle span creation\n", "\n", " Returns\n", " -------\n", " str\n", " JSON formatted string of the user payload with the updated response from the OpenAI API\n", " \"\"\"\n", " with tracer.start_as_current_span(\"Classify User Intent\") as span: # Define Span Name & Start\n", " user_payload_dict = json.loads(user_payload_json)\n", " customer_input = user_payload_dict.get(\"Customer Input\", \"\")\n", " response_dict = call_openai_api(user_prompt, customer_input)\n", " user_payload_dict.update(response_dict)\n", "\n", " # Define Custom Attribute String - Customer ID\n", " span.set_attribute(\"customerID.name\", user_payload_dict[\"Customer ID\"])\n", " # Define Custom Attribute String - Customer Input\n", " span.set_attribute(\"customerInput.name\", user_payload_dict[\"Customer Input\"])\n", " # Define Custom Attribute String - Premium Customer Bool String\n", " span.set_attribute(\"premiumCustomer.name\", user_payload_dict[\"Premium Customer\"])\n", "\n", " # Define Span Type as \"CHAIN\"\n", " span.set_attribute(\n", " SpanAttributes.OPENINFERENCE_SPAN_KIND, OpenInferenceSpanKindValues.CHAIN.value\n", " )\n", "\n", " # Set Status Code\n", " span.set_status(trace_api.StatusCode.OK)\n", "\n", " return json.dumps(user_payload_dict)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "def item_search(\n", " user_payload_json: str,\n", " items_df: pd.DataFrame,\n", " tracer: opentelemetry.sdk.trace.Tracer,\n", ") -> str:\n", " \"\"\"Create span of the search for a purchase item\n", "\n", " Parameters\n", " ----------\n", " user_payload_json : str\n", " Item payload JSON string\n", " items_df : pd.DataFrame\n", " DataFrame containting item data\n", " tracer : opentelemetry.sdk.trace.Tracer\n", " Tracer to handle span creation\n", "\n", " Returns\n", " -------\n", " str\n", " JSON formatted string of the item payload\n", " \"\"\"\n", " # Define Span Name & Start\n", " with tracer.start_as_current_span(\"Search for Purchase Item\") as span:\n", " trace_api.get_current_span()\n", " user_payload_dict = json.loads(user_payload_json)\n", " updated_dict = update_payload_with_search_results(\n", " user_payload_dict, items_df, \"Best Use\", \"shopping_category\"\n", " )\n", "\n", " # Define Custom Attribute String - Shopping Category String\n", " span.set_attribute(\"shopping_category.name\", updated_dict[\"shopping_category\"])\n", "\n", " # Define Custom Attribute String - Item String\n", " span.set_attribute(\"Item.name\", updated_dict[\"Item\"])\n", "\n", " # Define Custom Attribute Value - Stars Value\n", " span.set_attribute(\"Stars.value\", updated_dict[\"Stars\"])\n", "\n", " # Define Span Type as \"CHAIN\"\n", " span.set_attribute(\n", " SpanAttributes.OPENINFERENCE_SPAN_KIND, OpenInferenceSpanKindValues.CHAIN.value\n", " )\n", "\n", " # Set Status Code\n", " span.set_status(trace_api.StatusCode.OK)\n", "\n", " return json.dumps(updated_dict)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "def answer_search(\n", " user_payload_json: str,\n", " policy_df: pd.DataFrame,\n", " tracer: opentelemetry.sdk.trace.Tracer,\n", ") -> str:\n", " \"\"\"If customer intent is an inquiry, search for the answer in the policy data\n", "\n", " Parameters\n", " ----------\n", " user_payload_json : str\n", " JSON formatted string of the user payload\n", " policy_df : pd.DataFrame\n", " Dataframe of policy data\n", " tracer : opentelemetry.sdk.trace.Tracer\n", " Tracer to handle span creation\n", "\n", " Returns\n", " -------\n", " str\n", " JSON formatted string of the answer payload\n", " \"\"\"\n", " # Define Span Name & Start\n", " with tracer.start_as_current_span(\"Search for Query Answer\") as span:\n", " user_payload_dict = json.loads(user_payload_json)\n", " updated_dict = update_payload_with_search_results(\n", " user_payload_dict, policy_df, \"Category\", \"query_category\"\n", " )\n", "\n", " keys_to_update = {\"Question\", \"Answer\"}\n", " updated_dict = {\n", " k: v for k, v in updated_dict.items() if k in keys_to_update or k in user_payload_dict\n", " }\n", "\n", " # Define Custom Attribute String - Shopping Category String\n", " span.set_attribute(\"query_category.name\", updated_dict[\"Category\"])\n", " # Define Define Custom Attribute String - Query Text String\n", " span.set_attribute(\"query_text.name\", updated_dict[\"Question\"])\n", " # Define Define Custom Attribute String - Reference Text String\n", " span.set_attribute(\"reference_text.name\", updated_dict[\"Answer\"])\n", "\n", " # Define Span Type as \"CHAIN\"\n", " span.set_attribute(\n", " SpanAttributes.OPENINFERENCE_SPAN_KIND, OpenInferenceSpanKindValues.CHAIN.value\n", " )\n", "\n", " # Set Status Code\n", " span.set_status(trace_api.StatusCode.OK)\n", "\n", " return json.dumps(updated_dict)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "def item_search_response(\n", " user_payload_json: str,\n", " item_search_prompt: str,\n", " tracer: opentelemetry.sdk.trace.Tracer,\n", ") -> str:\n", " \"\"\"Query response when customer asks a purchase question\n", "\n", " Parameters\n", " ----------\n", " user_payload_json : str\n", " JSON formatted string for prompt template input\n", " item_search_prompt : str\n", " Item search prompt template\n", " tracer : opentelemetry.sdk.trace.Tracer\n", " Tracer to handle span creation\n", "\n", " Returns\n", " -------\n", " str\n", " JSON formatted string of item search payload\n", " \"\"\"\n", " # Define Span Name & Start\n", " with tracer.start_as_current_span(\"Item Search Response\") as span:\n", " user_payload_dict = json.loads(user_payload_json)\n", " customer_input = user_payload_dict.get(\"Customer Input\", \"\")\n", " response_dict = call_openai_api(item_search_prompt, customer_input)\n", " user_payload_dict.update(response_dict)\n", "\n", " # Define Span Type as \"CHAIN\"\n", " span.set_attribute(\n", " SpanAttributes.OPENINFERENCE_SPAN_KIND, OpenInferenceSpanKindValues.CHAIN.value\n", " )\n", "\n", " # Set Status Code\n", " span.set_status(trace_api.StatusCode.OK)\n", "\n", " return json.dumps(user_payload_dict)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "def query_search_response(\n", " user_payload_json: str,\n", " customer_qa_prompt: str,\n", " tracer: opentelemetry.sdk.trace.Tracer,\n", ") -> str:\n", " \"\"\"Query response when customer has an inquiry\n", "\n", " Parameters\n", " ----------\n", " user_payload_json : str\n", " JSON formatted string for Q&A prompt template input\n", " customer_qa_prompt : str\n", " Customer Q&A prompt template\n", " tracer : opentelemetry.sdk.trace.Tracer\n", " Tracer to handle span creation\n", "\n", " Returns\n", " -------\n", " str\n", " JSON formatted string of query search payload\n", " \"\"\"\n", " # Define Span Name & Start\n", " with tracer.start_as_current_span(\"Query Search Response\") as span:\n", " user_payload_dict = json.loads(user_payload_json)\n", " customer_input = user_payload_dict.get(\"Customer Input\", \"\")\n", " response_dict = call_openai_api(customer_qa_prompt, customer_input)\n", " user_payload_dict.update(response_dict)\n", "\n", " # Define Span Type as \"CHAIN\"\n", " span.set_attribute(\n", " SpanAttributes.OPENINFERENCE_SPAN_KIND, OpenInferenceSpanKindValues.CHAIN.value\n", " )\n", "\n", " # Set Status Code\n", " span.set_status(trace_api.StatusCode.OK)\n", "\n", " return json.dumps(user_payload_dict)" ] }, { "cell_type": "markdown", "metadata": { "id": "4-Uu5OEj0gBL" }, "source": [ "#### Helper Functions\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "def call_openai_api(user_prompt: str, user_input: str) -> dict:\n", " \"\"\"Issue requests to the OpenAI API\n", "\n", " Parameters\n", " ----------\n", " user_prompt : str\n", " Prompt template for OpenAI API\n", " user_input : str\n", " Prompt input for OpenAI API\n", "\n", " Returns\n", " -------\n", " dict\n", " Dictionary of response from OpenAI API\n", " \"\"\"\n", " client = OpenAI()\n", " try:\n", " response = client.chat.completions.create(\n", " model=\"gpt-3.5-turbo\",\n", " response_format={\"type\": \"json_object\"},\n", " messages=[\n", " {\"role\": \"system\", \"content\": user_prompt},\n", " {\"role\": \"user\", \"content\": user_input},\n", " ],\n", " )\n", " return json.loads(response.choices[0].message.content)\n", " except Exception as e:\n", " print(f\"Error calling OpenAI API: {e}\")\n", " return {}\n", "\n", "\n", "def update_payload_with_search_results(\n", " user_payload_dict: dict, search_df: pd.DataFrame, search_column: str, match_key: str\n", ") -> dict:\n", " \"\"\"Update user payload with search results\n", "\n", " Parameters\n", " ----------\n", " user_payload_dict : dict\n", " Dictionary of user data key-value pairs\n", " search_df : pd.DataFrame\n", " DataFrame of of item or policy data\n", " search_column : str\n", " Column to subset DataFrame\n", " match_key : str\n", " Key name to query in user_payload_dict\n", "\n", " Returns\n", " -------\n", " dict\n", " Dictionary of updated user payload\n", " \"\"\"\n", " search_key_value = user_payload_dict.get(match_key, \"\")\n", " matching_row = search_df[search_df[search_column] == search_key_value].iloc[0].to_dict()\n", " user_payload_dict.update(matching_row)\n", " return user_payload_dict\n", "\n", "\n", "def pretty_print_result(result_dict: dict) -> str:\n", " \"\"\"Format the output results\n", "\n", " Parameters\n", " ----------\n", " result_dict : dict\n", " Dictionary of results\n", "\n", " Returns\n", " -------\n", " str\n", " String of key and value pairs\n", " \"\"\"\n", " for key, value in result_dict.items():\n", " print(f\"{key}: {value}\")\n", " print(f\"\\n{'-' * 50}\\n\")" ] }, { "cell_type": "markdown", "metadata": { "id": "RI9cIPKpfDGx" }, "source": [ "## Run LLM Application\n", "\n", "Once all functions are defined, we will call them within `run_llm_app`, a centralized function.\n", "\n", "As the function runs per query, note tracing data will populate within Phoenix." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "def run_llm_app(\n", " row_json: str, customer_intent_prompt: str, tracer: opentelemetry.sdk.trace.Tracer\n", ") -> dict:\n", " \"\"\"Run manual instrumentation of the LLM application\n", "\n", " Parameters\n", " ----------\n", " row_json : str\n", " JSON formatted string of row data\n", " customer_intent_prompt : str\n", " Customer intent prompt (is customer asking a about purchases or a separate query)\n", " tracer : opentelemetry.sdk.trace.Tracer\n", " Tracer to handle span creation\n", "\n", " Returns\n", " -------\n", " dict\n", " Dictionary of response results\n", " \"\"\"\n", " # Define Span Name & Start\n", " with tracer.start_as_current_span(\"Customer Session\") as span:\n", " # Define Open Inference Semanantic Convention - Input\n", " span.set_attribute(\"input.value\", row[\"Customer Input\"])\n", "\n", " if not isinstance(row_json, str):\n", " row_json = row_json.to_json()\n", "\n", " intent_response_json = openai_classify_user_intent(\n", " customer_intent_prompt, row_json, tracer=tracer\n", " )\n", " intent_response_dict = json.loads(intent_response_json)\n", "\n", " intent = intent_response_dict.get(\"customer_intent\")\n", " if intent == \"purchase\":\n", " result_purchase_json = item_search(intent_response_json, items_df, tracer=tracer)\n", " result_purchase_dict = json.loads(result_purchase_json)\n", " return_result_response_json = item_search_response(\n", " json.dumps(result_purchase_dict), item_search_prompt, tracer=tracer\n", " )\n", "\n", " elif intent == \"query\":\n", " result_query_json = answer_search(intent_response_json, policy_df, tracer=tracer)\n", " result_query_dict = json.loads(result_query_json)\n", " return_result_response_json = query_search_response(\n", " json.dumps(result_query_dict), customer_qa_prompt, tracer=tracer\n", " )\n", "\n", " else:\n", " return_result_response_json = json.dumps(\n", " {\n", " \"message\": \"Sorry, I couldn't help out. Please reach out to support for more help.\"\n", " }\n", " )\n", "\n", " result_response_dict = json.loads(return_result_response_json)\n", "\n", " # Define Open Inference Semanantic Convention - Output\n", " span.set_attribute(\"output.value\", result_response_dict[\"customer_response\"])\n", "\n", " # Define Custom Attribute String - Customer ID\n", " span.set_attribute(\"customerID.name\", result_response_dict[\"Customer ID\"])\n", "\n", " # Define Custom Attribute String - Premium Customer Bool String\n", " span.set_attribute(\"premiumCustomer.name\", result_response_dict[\"Premium Customer\"])\n", "\n", " # Define Span Type as \"CHAIN\"\n", " span.set_attribute(\n", " SpanAttributes.OPENINFERENCE_SPAN_KIND, OpenInferenceSpanKindValues.CHAIN.value\n", " )\n", "\n", " # Set Status Code\n", " span.set_status(trace_api.StatusCode.OK)\n", "\n", " return result_response_dict" ] }, { "cell_type": "markdown", "metadata": { "id": "ogcSgcD4MFEf" }, "source": [ "## Submit Queries to LLM\n", "\n", "Once `run_llm_app` is defined, the cell below does the following:\n", "* Convert each DataFrame row to JSON format\n", "* Runs the LLM application per row, and populate within Phoenix\n", "* Formats output in the notebook" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "for _, row in customer_inputs_df.iterrows():\n", " row_json = row.to_json()\n", " result = run_llm_app(row_json, customer_intent_prompt, tracer=tracer)\n", " pretty_print_result(result)" ] }, { "cell_type": "markdown", "metadata": { "id": "ScUuL5Xd2cGF" }, "source": [ "## Additional Resources - Attributes\n", "\n", "As your adding attributes make sure to visit the [Semantic Conventions documentation](https://github.com/Arize-ai/openinference/blob/main/spec/semantic_conventions.md) - this will tell you what you will likely define from an attribute perspective\n", "\n", "For custom attributes, visit the [Custom Spans documentation](https://arize.com/docs/phoenix/tracing/how-to-tracing/custom-spans#add-attributes-to-a-span). This will help you to define custom attributes and other data outside of the Open Inference Semantic Conventions" ] } ], "metadata": { "language_info": { "name": "python" } }, "nbformat": 4, "nbformat_minor": 4 }