TxtAI MCP Server

{ "cells": [ { "cell_type": "markdown", "metadata": { "id": "vwELCooy4ljr" }, "source": [ "# Prompt templates and task chains\n", "\n", "txtai has long had support for workflows. Workflows connect the input and outputs of machine learning models together to create powerful transformation and processing functions.\n", "\n", "There has been a recent surge in interest in \"model prompting\", which is the process of building a natural language description of a task and passing it to a large language model (LLM). txtai has recently improved support for task templating, which builds string outputs from a set of parameters.\n", "\n", "This notebook demonstrates how txtai workflows can be used to apply prompt templates and chain those tasks together." ] }, { "cell_type": "markdown", "metadata": { "id": "ew7orE2O441o" }, "source": [ "# Install dependencies\n", "\n", "Install `txtai` and all dependencies." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "LPQTb25tASIG" }, "outputs": [], "source": [ "%%capture\n", "!pip install git+https://github.com/neuml/txtai#egg=txtai[api]" ] }, { "cell_type": "markdown", "metadata": { "id": "_YnqorRKAbLu" }, "source": [ "# Prompt workflow\n", "\n", "First, we'll look at building a workflow with a series of model prompts. This workflow creates a conditional translation using a statement and target language. Another task reads that output text and detects the language.\n", "\n", "This workflow uses a LLM pipeline. The LLM pipeline loads a local model for inference, in this case [Qwen3-4B-Instruct-2507](https://huggingface.co/Qwen/Qwen3-4B-Instruct-2507). The [LLM pipeline](https://neuml.github.io/txtai/pipeline/llm/llm) supports local transformers models, llama.cpp models and LLM APIs such as Ollama, vLLM, OpenAI, Claude etc. \n", "\n", "It's important to note that a pipeline is simply a callable function. It can easily be replaced with a call to an external API." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "OUc9gqTyAYnm", "outputId": "83300311-736c-47c8-bc16-ec0303274054" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "['French', 'German']\n" ] } ], "source": [ "from txtai import LLM, Workflow\n", "from txtai.workflow import TemplateTask\n", "\n", "# Create LLM\n", "llm = LLM(\"Qwen/Qwen3-4B-Instruct-2507\")\n", "\n", "# Define workflow or chaining of tasks together.\n", "workflow = Workflow([\n", " TemplateTask(\n", " template=\"Translate text '{statement}' to {language} if the text is English, otherwise keep the original text\",\n", " action=llm\n", " ),\n", " TemplateTask(\n", " template=\"What language is the following text. Only print the answer? {text}\",\n", " action=llm\n", " )\n", "])\n", "\n", "inputs = [\n", " {\"statement\": \"Hello, how are you\", \"language\": \"French\"},\n", " {\"statement\": \"Hallo, wie geht's dir\", \"language\": \"French\"}\n", "]\n", "\n", "print(list(workflow(inputs)))" ] }, { "cell_type": "markdown", "metadata": { "id": "_zz4Do8BV-Lk" }, "source": [ "Let's recap what happened here. The first workflow task conditionally translates text to a language if it's English.\n", "\n", "The first statement is `Hello, how are you` with a target language of French. So the statement is translated to French.\n", "\n", "The second statement is German, so it's not converted to French.\n", "\n", "The next step asks the model what the language is and it correctly prints `French` and `German`." ] }, { "cell_type": "markdown", "metadata": { "id": "iXDAKP4CX0W9" }, "source": [ "# Prompt Workflow as YAML\n", "\n", "The same workflow above can be created with YAML configuration." ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "GwV5A9xRYtYs", "outputId": "ffe6ee65-95a7-46c6-e6b9-5324eab26ca8" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Writing workflow.yml\n" ] } ], "source": [ "%%writefile workflow.yml\n", "\n", "llm:\n", " path: Qwen/Qwen3-4B-Instruct-2507\n", "\n", "workflow:\n", " chain:\n", " tasks:\n", " - task: template\n", " template: Translate text '{statement}' to {language} if the text is English, otherwise keep the original text\n", " action: llm\n", " - task: template\n", " template: What language is the following text. Only print the answer? {text}\n", " action: llm" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "dr7Lv5S5X98e", "outputId": "d6ac0427-671d-4525-aa21-664430109af3" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "['French', 'German']\n" ] } ], "source": [ "from txtai import Application\n", "\n", "app = Application(\"workflow.yml\")\n", "print(list(app.workflow(\"chain\", inputs)))" ] }, { "cell_type": "markdown", "metadata": { "id": "EGqiV45fYVse" }, "source": [ "As expected, the same result! This is a matter of preference on how you want to create a workflow. One advantage of YAML workflows is that an API can easily be created from the workflow file." ] }, { "cell_type": "markdown", "metadata": { "id": "9PqMU0bNYinf" }, "source": [ "# Prompt Workflow via an API call\n", "\n", "Let's say you want the workflow to be available via an API call. Well good news, txtai has a built in API mechanism using FastAPI. " ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "vDxQj1ZIYsz3" }, "outputs": [], "source": [ "# Start an API service\n", "!CONFIG=workflow.yml nohup uvicorn \"txtai.api:app\" &> api.log &\n", "!sleep 60" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "R1o08SVtZW7h", "outputId": "99875acd-18a8-4c2c-ead3-cb6975a4b2d2" }, "outputs": [ { "data": { "text/plain": [ "['French', 'German']" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import requests\n", "\n", "# Run API request\n", "requests.post(\"http://localhost:8000/workflow\", json={\"name\": \"chain\", \"elements\": inputs}).json()" ] }, { "cell_type": "markdown", "metadata": { "id": "B88mCrGFl5W-" }, "source": [ "Just like the previous steps, except through an API call. Let's run via cURL for good measure." ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "hRUyh0cQl_P2", "outputId": "9db8481d-0b6e-4a31-bdf6-5443df5f768a" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[\"French\",\"German\"]" ] } ], "source": [ "%%bash\n", "\n", "curl -s -X POST \"http://localhost:8000/workflow\" \\\n", " -H \"Content-Type: application/json\" \\\n", " --data @- << EOF\n", "{\n", " \"name\": \"chain\",\n", " \"elements\": [\n", " {\"statement\": \"Hello, how are you\", \"language\": \"French\"},\n", " {\"statement\": \"Hallo, wie geht's dir\", \"language\": \"French\"}\n", " ]\n", "}\n", "EOF" ] }, { "cell_type": "markdown", "metadata": { "id": "W0zL93WPoaCo" }, "source": [ "One last time, the same output is shown.\n", "\n", "If your primary development environment isn't Python, txtai does have API bindings for [JavaScript](https://github.com/neuml/txtai.js), [Rust](https://github.com/neuml/txtai.rs), [Go](https://github.com/neuml/txtai.go) and [Java](https://github.com/neuml/txtai.java).\n", "\n", "More information on the API is available [here](https://neuml.github.io/txtai/api/)." ] }, { "cell_type": "markdown", "metadata": { "id": "q9WiFG6fpzw5" }, "source": [ "# Chat with your data\n", "\n", "\"Chat with your data\" is a popular entry point into the AI space. Let's run an example." ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "rM3Y551LqF-J", "outputId": "85623785-c15f-4996-9460-0644f69cf5bf" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Writing search.yml\n" ] } ], "source": [ "%%writefile search.yml\n", "\n", "writable: false\n", "cloud:\n", " provider: huggingface-hub\n", " container: neuml/txtai-intro\n", "\n", "rag:\n", " path: Qwen/Qwen3-4B-Instruct-2507\n", " output: reference\n", " template: |\n", " Answer the following question using only the context below.\n", "\n", " Question: {question}\n", " Context: {context}\n", "\n", "workflow:\n", " search:\n", " tasks:\n", " - action: rag\n", " - task: template\n", " template: \"{answer}\\n\\nReference: {reference}\"\n", " rules:\n", " answer: I don't have data on that" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "1Elb8JANqpwX", "outputId": "b1f1ffa1-6c47-4d90-b6f1-8098d4dc45f8" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[\"Canada's last fully intact ice shelf has suddenly collapsed, forming a Manhattan-sized iceberg.\\n\\nReference: 1\"]\n" ] } ], "source": [ "app = Application(\"search.yml\")\n", "print(list(app.workflow(\"search\", [\"Find something about North America\"])))" ] }, { "cell_type": "markdown", "metadata": { "id": "4r49V4c9s5nf" }, "source": [ "The first thing the code above does is run an embeddings search to build a conversational context. That context is then used to build a prompt and inference is run against the LLM. \n", "\n", "The next task formats the outputs with a reference to the best matching record. In this case, it's only an id of 1. But this can be much more useful if the id is a URL or there is logic to format the id back to a unique reference string." ] }, { "cell_type": "markdown", "metadata": { "id": "KqfvCXp2B3li" }, "source": [ "# Wrapping up\n", "\n", "This notebook covered how to build prompt templates and task chains through a series of results. txtai has long had a robust and efficient workflow framework for connecting models together. This can be small and simple models and/or prompting with large models. Go ahead and give it a try!" ] } ], "metadata": { "accelerator": "GPU", "colab": { "provenance": [] }, "gpuClass": "standard", "kernelspec": { "display_name": "local", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.19" } }, "nbformat": 4, "nbformat_minor": 0 }