Medical GraphRAG Assistant

{ "cells": [ { "cell_type": "markdown", "id": "6d2ee568-cc16-4f1e-a3f1-3ca6cb8d7f44", "metadata": {}, "source": [ "# Adding FHIR data to a FHIR server\n", "\n", "Your project may involve ingesting data into FHIR format and sending it to a FHIR server. This notebook is a quick example of how this can be achieved with IRIS health. \n", "\n", "This tutorial will assume you have a FHIR server set up with IRIS-health as described in the [FHIR Server quickstart tutorial](../Tutorial/0-FHIR-server-setup.md). If you have set up the FHIR server by different methods, the differences will be where the requests are send (Endpoint URL, username and password) but the content will be the same. \n" ] }, { "cell_type": "markdown", "id": "1b243ee5", "metadata": {}, "source": [ "## Add existing FHIR files to the FHIR server\n", "\n", "You can also add a bundle of FHIR resources to a FHIR Server using HTTP requests - the FHIR server will act the resources to the correct place. For an example, I am going to add 5 patient bundles - that is the complete medical history of the patient. These have been synthetically generated by Synthea. The instructions for this are a [separate tutorial](Creating_synthetic_FHIR-data.md), but are super simple - use the command: \n", "\n", " docker run --rm -v $PWD/output:/output --name synthea-docker intersystemsdc/irisdemo-base-synthea:version-1.3.4 -p 5\n", "\n", "Where -p denotes the number of patients to generate.\n", "\n", "First, lets look at the files: " ] }, { "cell_type": "code", "execution_count": null, "id": "798cba5e", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['hospitalInformation1760013140357.json',\n", " 'Jarred626_Walsh511_ea6f7f29-8553-4456-8fa9-3d629ffcd5f7.json',\n", " 'Krista314_Mosciski958_79fdc511-4556-435d-b922-7a55d0d6e409.json',\n", " 'Kristofer887_Schowalter414_a7b87618-d13a-4f02-8974-010aa35fb759.json',\n", " 'Marshall526_Mosciski958_dbc8773a-10ac-42e5-9c8c-c8d269ed9179.json',\n", " 'Milissa240_Gaylord332_db7fc62f-0f83-484f-bf84-40e85fc26c4d.json',\n", " 'practitionerInformation1760013140357.json']" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "import os\n", "os.listdir(\"./output/fhir\")" ] }, { "cell_type": "code", "execution_count": null, "id": "dc6e1944", "metadata": {}, "outputs": [], "source": [ "import json\n", "import requests\n", "from requests.auth import HTTPBasicAuth\n", "\n", "server_url = \"http://localhost:32783/csp/healthshare/demo/fhir/r4/\"\n", "username = \"_SYSTEM\"\n", "password = \"ISCDEMO\"\n", "\n", "#Iterate over the files\n", "for file in os.listdir(\"./output/fhir\"):\n", " # Read the File\n", " with open(\"./output/fhir/\"+file, \"r\")as f:\n", " json_data = json.load(f)\n", " \n", " res = requests.post( server_url,\n", " json=json_data,\n", " headers={\"Content-Type\": \"application/fhir+json\"}, \n", " auth=HTTPBasicAuth(username, password))\n", " print(res)\n", " if res.status_code!=200:\n", " print(res.json())" ] }, { "cell_type": "markdown", "id": "3057ba19", "metadata": {}, "source": [ "Lets just check the patient resources were added: " ] }, { "cell_type": "code", "execution_count": null, "id": "f3d4a43a", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "<Response [200]>\n", "3615 [{'use': 'official', 'family': 'Walsh511', 'given': ['Jarred626']}] male\n", "3878 [{'use': 'official', 'family': 'Mosciski958', 'given': ['Krista314'], 'prefix': ['Mrs.']}, {'use': 'maiden', 'family': 'Frami345', 'given': ['Krista314'], 'prefix': ['Mrs.']}] female\n", "4363 [{'use': 'official', 'family': 'Schowalter414', 'given': ['Kristofer887'], 'prefix': ['Mr.']}] male\n", "4510 [{'use': 'official', 'family': 'Mosciski958', 'given': ['Marshall526'], 'prefix': ['Mr.']}] male\n", "5072 [{'use': 'official', 'family': 'Gaylord332', 'given': ['Milissa240'], 'prefix': ['Ms.']}] female\n" ] } ], "source": [ "res = requests.get(baseURL+endpoint, headers=get_headers, auth=HTTPBasicAuth(username, password))\n", "print(res)\n", "#print(res.json())\n", "data = res.json() \n", "\n", "for entry in data.get(\"entry\", [])[-5:]: ## Get the last 5 entries\n", " patient = entry.get('resource', {}) ## Get the patient resource\n", " print(patient.get('id'), patient.get('name'), patient.get('gender')) ## Print the patient info\n" ] }, { "cell_type": "markdown", "id": "a539d859", "metadata": {}, "source": [ "And there we have it! How to add FHIR resources into a FHIR server running with InterSystems IRIS for Health. I hope you have found this useful! " ] }, { "cell_type": "markdown", "id": "5d99e003", "metadata": {}, "source": [ "\n", "## Adding New FHIR data to a FHIR Server\n", "\n", "I am going to add FHIR data to the server in two steps, firstly I am going to create a valid fhir resource - the python package [`fhir.resources`](https://github.com/nazrulworld/fhir.resources) is going to help with this. Then I will send a POST request to the server using the standard http `requests` library. \n", "\n", "### Step 1: Create valid resources\n", "\n", "First of all, install the dependancy - this is [fhir.resources](https://github.com/nazrulworld/fhir.resources). There is much more information about its usage on the linked repo than I am going to include here, so check it out if you want more information\n" ] }, { "cell_type": "code", "execution_count": null, "id": "3488307d-78fc-43bb-a956-d85348c70cbd", "metadata": {}, "outputs": [], "source": [ "pip install fhir.resources " ] }, { "cell_type": "markdown", "id": "ebf17f4f-e4e9-40c0-9634-812d7ba528e7", "metadata": {}, "source": [ "You then need to import the type of resource you would like to make. I am going to start by creating a new patient using a JSON format object:" ] }, { "cell_type": "code", "execution_count": 25, "id": "2d678828-acd1-4325-9f3f-2a1c296a1c46", "metadata": {}, "outputs": [], "source": [ "from fhir.resources.patient import Patient" ] }, { "cell_type": "code", "execution_count": 115, "id": "8096ab27-b60e-4b69-a6fd-9721b6e65aa6", "metadata": {}, "outputs": [], "source": [ "data = { ## Create a data object that resembles a fhir resource\n", " \"name\": [{\n", " \"use\": \"official\",\n", " \"family\": \"Kent\", \n", " \"given\": [\"Clark\"] ## As you can have multiple given names (middle names) this needs to be a list item\n", " }],\n", " \"birthDate\":\"1965-02-12\", \"gender\":\"male\"}\n", "superman = Patient.model_validate(data)" ] }, { "cell_type": "code", "execution_count": 103, "id": "2d992d11-f9be-4ae2-ba08-7566749c213d", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "1965-02-12\n", "[HumanName(fhir_comments=None, extension=None, id=None, family='Kent', family__ext=None, given=['Clark'], given__ext=None, period=None, prefix=None, prefix__ext=None, suffix=None, suffix__ext=None, text=None, text__ext=None, use='official', use__ext=None)]\n" ] } ], "source": [ "print(superman.birthDate)\n", "print(superman.name)" ] }, { "cell_type": "markdown", "id": "23c7d022-b1f3-4db5-8216-fadd59cbe458", "metadata": {}, "source": [ "Alternatively we can use a more Pythonic type way:" ] }, { "cell_type": "code", "execution_count": 116, "id": "41692d4c-ca59-4d9d-90e2-5ebbbd1e363c", "metadata": {}, "outputs": [], "source": [ "batman = Patient.model_construct()\n", "\n", "\n", "batman.name = [{}] # The name part of the patient resource is a bit of a pain! \n", "batman.id = \"500\"\n", "batman.name[0].family = \"Wayne\"\n", "batman.name[0].given = [\"Bruce\"]\n", "batman.birthDate = \"1982-01-15\"\n", "batman.gender= \"male\"" ] }, { "cell_type": "markdown", "id": "13620693-5899-4132-8eab-ed697a50820d", "metadata": {}, "source": [ "#### Accesing model data" ] }, { "cell_type": "code", "execution_count": 105, "id": "cd116d3c-10cd-4454-8f26-fee49e33f023", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{\"resourceType\":\"Patient\",\"id\":\"500\",\"name\":[{\"family\":\"Wayne\",\"given\":[\"Bruce\"]}],\"gender\":\"Male\",\"birthDate\":\"1982-01-15\"} \n", " <class 'str'> \n", "\n", "\n", "{'resourceType': 'Patient', 'name': [{'use': 'official', 'family': 'Kent', 'given': ['Clark']}], 'gender': 'Male', 'birthDate': datetime.date(1965, 2, 12)} \n", " <class 'dict'>\n" ] } ], "source": [ "batman_model = batman.model_dump_json() \n", "print(batman_model,'\\n' , type(batman_model), '\\n\\n') ## This returns a JSON string\n", "\n", "superman_model = superman.model_dump() ## This returns a json object (or python dict)\n", "\n", "print(superman_model, '\\n', type(superman_model))" ] }, { "cell_type": "markdown", "id": "bbdc7611-61fb-48a6-9d99-d7e09dc88717", "metadata": {}, "source": [ "There are classes for every fhir resource defined in the FHIR specification, so a similar method can be used with any resource you may need - Here I'm going to create a DocumentReference resource. If you have worked through the main demo in this repository, you may be familiar with this resource type, but it can be used to link any different type of documents. Here I am going to load a premade clinical note into a document reference: \n" ] }, { "cell_type": "code", "execution_count": 164, "id": "bb5e0d7c-d163-42cd-84a9-dafe00befafa", "metadata": {}, "outputs": [], "source": [ "from fhir.resources.documentreference import DocumentReference\n", "from datetime import datetime\n", "note_resource = DocumentReference.model_construct()\n", "\n", "\n", "with open(\"note_data/clinical_note.txt\", \"r\", encoding=\"utf-8\") as f:\n", " note = f.read()\n", " \n", "\n", "note_resource.status = \"current\"\n", "note_resource.date = datetime(2025,9,10)\n", "note_resource.subject = {\"reference\": \"500\"}\n", "note_resource.content = [{ \"attachment\":{\"contentType\": \"text/plain; charset=utf-8\", \"data\": note.encode(\"utf-8\").hex()}}]\n" ] }, { "cell_type": "markdown", "id": "5424208b-69ef-4cce-9854-53ee55e7cc9a", "metadata": {}, "source": [ "### Step 2: Adding data to the server\n", "\n", "Similar to how we accessed the data with http requests in the [Accessing-FHIR-resources](./Accessing-FHIR-resources.ipynb) demo, we can add the FHIR resources we have created to the server using HTTP requests. We do need to add basic authorisation. " ] }, { "cell_type": "code", "execution_count": 120, "id": "1a59f8fe-b8ad-4022-af10-1fccebf8c05b", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "<Response [201]>\n" ] } ], "source": [ "import requests\n", "from requests.auth import HTTPBasicAuth\n", "\n", "## Credentials\n", "username = \"_System\"\n", "password = \"ISCDEMO\"\n", "\n", "## FHIR server location\n", "baseURL = \"http://localhost:32783/csp/healthshare/demo/fhir/r4/\"\n", "headers ={ \"Content-Type\": \"application/fhir+json\"}\n", "endpoint = \"Patient\"\n", "\n", "\n", "res = requests.post(baseURL+endpoint, \n", " data=superman.model_dump_json(), ## Give a JSON string as data - giving it a Python dict may cause errors because of the datetime object.\n", " headers= headers, ## We are defining the content type (FHIR+JSON) in the header\n", " auth=HTTPBasicAuth(username, password))\n", "print(res)" ] }, { "cell_type": "code", "execution_count": null, "id": "cb56dc36-ab6f-4b8f-baaa-f5be3a11e09b", "metadata": {}, "outputs": [], "source": [ "## If you have any error message, you can find some more information with \n", "print(res.json())\n", "## This will error if the data has been added successfully though!" ] }, { "cell_type": "markdown", "id": "94f3853c-a59a-4458-af6f-6e27057916a4", "metadata": {}, "source": [ "Response 201 means it has been successfully added!" ] }, { "cell_type": "code", "execution_count": 117, "id": "6b2e4bd0-d3eb-434a-9a56-893475438a43", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "<Response [201]>\n" ] } ], "source": [ "res = requests.post(baseURL+endpoint, data= batman.model_dump_json(), headers= headers, auth=HTTPBasicAuth(username, password))\n", "\n", "print(res)" ] }, { "cell_type": "markdown", "id": "6460dc7e-72b3-448a-a495-343802f61a2c", "metadata": {}, "source": [ "#### Checking data is added\n", "Lets just query the endpoint to see whether we have added them correctly:" ] }, { "cell_type": "code", "execution_count": 148, "id": "0a75aa6b-a9b6-4453-b588-be7c27fcada5", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "<Response [200]>\n", "32 [{'use': 'official', 'family': 'Lemke', 'given': ['Tish'], 'prefix': ['Mrs.']}, {'use': 'maiden', 'family': 'Johnston', 'given': ['Tish'], 'prefix': ['Mrs.']}] female\n", "3596 [{'use': 'official', 'family': 'Kent', 'given': ['Clark']}] None\n", "3597 [{'use': 'official', 'family': 'Kent', 'given': ['Clark']}] None\n", "3598 [{'family': 'Wayne', 'given': ['Bruce']}] None\n", "3599 [{'family': 'Wayne', 'given': ['Bruce']}] male\n", "3600 [{'use': 'official', 'family': 'Kent', 'given': ['Clark']}] male\n" ] } ], "source": [ "## FHIR server location\n", "baseURL = \"http://localhost:32783/csp/healthshare/demo/fhir/r4/\"\n", "\n", "get_headers ={ \"Accept\": \"application/fhir+json\"}\n", "\n", "endpoint = \"Patient\"\n", "\n", "\n", "res = requests.get(baseURL+endpoint, headers=get_headers, auth=HTTPBasicAuth(username, password))\n", "print(res)\n", "#print(res.json())\n", "data = res.json() \n", "\n", "for entry in data.get(\"entry\", [])[5:]: ## Get the last 5 entries\n", " patient = entry.get('resource', {}) ## Get the patient resource\n", " print(patient.get('id'), patient.get('name'), patient.get('gender')) ## Print the patient info\n" ] }, { "cell_type": "markdown", "id": "1b9fc8e0-a46b-474f-9b93-db3dcf0f2271", "metadata": {}, "source": [ "So we have added them! Because we have used a POST request, they are automatically assigned an ID number. We could use a put request if we want to keep the ID number we've assigned them.\n" ] }, { "cell_type": "markdown", "id": "d00f1129-2298-48e8-b1a6-5c2fe2729db9", "metadata": {}, "source": [ "### Adding Document Reference\n", "\n", "Lets do the same with the document reference resource - only thing we need to change is the endpoint and the data:" ] }, { "cell_type": "code", "execution_count": 131, "id": "c8bd5138-c72c-476e-a030-e88a4bbe1f69", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "<Response [400]>\n", "{'resourceType': 'OperationOutcome', 'issue': [{'severity': 'error', 'code': 'invalid', 'diagnostics': '<HSFHIRErr>MalformedRelativeReference', 'details': {'text': \"The reference value '500' in property (subject) of Type 'DocumentReference' is malformed\"}, 'expression': ['DocumentReference.subject']}, {'severity': 'error', 'code': 'invalid', 'diagnostics': '<HSFHIRErr>MalformedValue', 'details': {'text': \"The value '2025-09-10T00:00:00' of Property 'date' of Type 'DocumentReference' is a malformed 'instant'. It should match the Regex '([0-9]([0-9]([0-9][1-9]|[1-9]0)|[1-9]00)|[1-9]000)-(0[1-9]|1[0-2])-(0[1-9]|[1-2][0-9]|3[0-1])T([01][0-9]|2[0-3]):[0-5][0-9]:([0-5][0-9]|60)(\\\\.[0-9]+)?(Z|(\\\\+|-)((0[0-9]|1[0-3]):[0-5][0-9]|14:00))'\"}, 'expression': ['DocumentReference.date']}]}\n" ] } ], "source": [ "res = requests.post(baseURL+\"DocumentReference\", data=note_resource.model_dump_json(),\n", " headers={\"Content-Type\": \"application/fhir+json\"}, auth=HTTPBasicAuth(username, password))\n", "\n", "print(res)\n", "print(res.json())" ] }, { "cell_type": "markdown", "id": "3a161174-23cd-4999-82f5-3216ae01aebd", "metadata": {}, "source": [ "Ok, so the fhir.resources doesnt completely validate our data - the FHIR server will reject bad data! Lets work through these issues: " ] }, { "cell_type": "code", "execution_count": 139, "id": "2810fd5c-1a47-46a0-9d0c-942fd3bb4a24", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "The reference value '500' in property (subject) of Type 'DocumentReference' is malformed\n", "\n", "\n", "The value '2025-09-10T00:00:00' of Property 'date' of Type 'DocumentReference' is a malformed 'instant'. It should match the Regex '([0-9]([0-9]([0-9][1-9]|[1-9]0)|[1-9]00)|[1-9]000)-(0[1-9]|1[0-2])-(0[1-9]|[1-2][0-9]|3[0-1])T([01][0-9]|2[0-3]):[0-5][0-9]:([0-5][0-9]|60)(\\.[0-9]+)?(Z|(\\+|-)((0[0-9]|1[0-3]):[0-5][0-9]|14:00))'\n", "\n", "\n" ] } ], "source": [ "for issue in res.json()[\"issue\"]:\n", " print(issue[\"details\"][\"text\"])\n", " print('\\n')" ] }, { "cell_type": "markdown", "id": "0602ce1c-6064-4cc8-8646-d6e2a72b741b", "metadata": {}, "source": [ "We can see two problems here - the patient reference is wrong, and the date is malformed because it doesnt have the timezone as a single letter value. Lets fix them and try again: " ] }, { "cell_type": "code", "execution_count": 143, "id": "6484672d-b155-4134-b6ef-404ccc267598", "metadata": {}, "outputs": [], "source": [ "note_resource.subject = {\"reference\": \"Patient/3600\"}\n", "note_resource.date = datetime(2025, 9, 10).isoformat() + \"Z\"\n" ] }, { "cell_type": "code", "execution_count": 145, "id": "9c17da2e-4dd7-483d-9779-77582ed3700c", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "<Response [201]>\n" ] } ], "source": [ "res = requests.post(baseURL+\"DocumentReference\", data=note_resource.model_dump_json(),\n", " headers={\"Content-Type\": \"application/fhir+json\"}, auth=HTTPBasicAuth(username, password))\n", "\n", "print(res)\n" ] }, { "cell_type": "code", "execution_count": 151, "id": "e00bfaca-6662-443e-8d82-79e80bf90c91", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "<Response [200]>\n" ] } ], "source": [ "res = requests.get(baseURL+\"DocumentReference?patient=3600\",\n", " headers=get_headers, auth=HTTPBasicAuth(username, password))\n", "print(res)" ] }, { "cell_type": "code", "execution_count": 163, "id": "baa8089d-da4f-4c97-b205-3f6920801bc6", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Patient Name: Clark Kent\n", "DOB: [Redacted]\n", "Date of Visit: 09/10/2025\n", "Presenting Complaint:\n", "Patient presents with complaints of persistent chest discomfort and intermittent episodes of spontaneous x-ray vision activation, particularly during emotionally charged situations (e.g., office meetings, romantic encounters, surprise birthday parties).\n", "History of Present Illness:\n", "Mr. Kent reports a recent increase in involuntary activation of his x-ray vision, resulting in several awkward social encounters, including accidentally seeing through three floors of the Daily Planet building. He also notes mild chest tightness after flying through a cumulonimbus cloud at Mach 3, which he describes as “like being hugged by a thunderstorm.”\n", "Past Medical History:\n", "\n", "Kryptonian physiology\n", "Solar overexposure syndrome (resolved)\n", "Mild allergic reaction to kryptonite (ongoing)\n", "\n", "Medications:\n", "\n", "None (immune to most Earth-based pharmaceuticals)\n", "\n", "Allergies:\n", "\n", "Kryptonite (severe photosensitivity, weakness, existential dread)\n", "\n", "Physical Examination:\n", "\n", "Vitals: Stable (heart rate 0 bpm, due to solar-powered circulatory system)\n", "Chest: Steel-like thoracic wall, no tenderness\n", "Eyes: Glowing intermittently, pupils reactive to red sun radiation\n", "Neurological: Alert, oriented, capable of hearing distress calls from 3 galaxies away\n", "\n", "Assessment:\n", "\n", "Hyperactive X-Ray Vision Syndrome (HXVS) – Likely exacerbated by emotional stress and overexposure to yellow sun radiation.\n", "Mild Thoracic Electromagnetic Burn – Secondary to high-speed atmospheric re-entry.\n", "\n", "Plan:\n", "\n", "Recommend temporary avoidance of high-emotion environments (e.g., Lois Lane’s apartment).\n", "Prescribe lead-lined sunglasses for vision control.\n", "Advise limiting solar charging to 2 hours/day.\n", "Follow-up in 1 week or sooner if symptoms worsen or if another alien invasion occurs.\n", "\n", "Physician:\n", "Dr. Emil Hamilton, MD, PhD, S.T.A.R. Labs\n" ] } ], "source": [ "resource = res.json()[\"entry\"][0][\"resource\"]\n", "# print(resource)\n", "print(bytes.fromhex(resource[\"content\"][0][\"attachment\"][\"data\"]).decode(\"utf-8\"))" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.11" } }, "nbformat": 4, "nbformat_minor": 5 }