From 3569829a1a2b939f05893d0021bfea5973a71cc5 Mon Sep 17 00:00:00 2001 From: Sophia Rowland Date: Thu, 27 Jun 2024 13:18:27 -0400 Subject: [PATCH 1/5] Adding new notebook for Azure Open AI example --- .../register_Azure_Open_AI_REST_Call.ipynb | 388 ++++++++++++++++++ 1 file changed, 388 insertions(+) create mode 100644 examples/register_Azure_Open_AI_REST_Call.ipynb diff --git a/examples/register_Azure_Open_AI_REST_Call.ipynb b/examples/register_Azure_Open_AI_REST_Call.ipynb new file mode 100644 index 00000000..5d99c16c --- /dev/null +++ b/examples/register_Azure_Open_AI_REST_Call.ipynb @@ -0,0 +1,388 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "3e928632-2b60-4eea-b296-1b196a29b3a4", + "metadata": {}, + "source": [ + "Copyright © 2024, SAS Institute Inc., Cary, NC, USA. All Rights Reserved. SPDX-License-Identifier: Apache-2.0\n", + "# Using Azure OpenAI GPT Models in SAS Viya\n", + "In this notebook, we will walk through the steps required for leveraging a GPT-3.5-Turbo model from Azure OpenAI in SAS® Model Manager® and SAS® Intelligent Decisioning®. \n", + "*** \n", + "## Table of Contents \n", + "1. [Introduction](#Introduction)\n", + "1. [Deploying a GPT Model in Azure OpenAI](#Deploying-a-GPT-Model-in-Azure-OpenAI)\n", + "1. [Considerations for Key Management](#Considerations-for-Key-Management)\n", + "1. [Integration with SAS Model Manager](#Integration-with-SAS-Model-Manager)\n", + "1. [Integration with SAS Intelligent Decisioning](#Integration-with-SAS-Intelligent-Decisioning)\n", + "1. [Conclusion](#Conclusion)\n", + "***\n", + "## Introduction \n", + "[Azure OpenAI Service](https://azure.microsoft.com/en-us/products/ai-services/openai-service/) provides access to Large Language Models (LLMs) through REST API, their Python SDK, and a web-based interface in the Azure OpenAI studio. Various models are available out of the box, including GPT-4, GPT-4 Turbo with Vision, GPT-3.5-Turbo, and Embeddings model series. You can read more about Azure OpenAI Service [here](https://learn.microsoft.com/en-us/azure/ai-services/openai/overview). To incorporate these models in SAS Intelligent Decisioning and SAS Model Manager, we will create a Python function and scoring code calling the REST API of a GPT-3.5-Turbo model deployed in Azure OpenAI. At the time of writing this notebook, the Python SDK does not support all calls to each model type, but that may change in the future. Additionally, we will walk through the steps required for deploying the model, based on what is required at the time of writing. \n", + "*** \n", + "## Deploying a GPT Model in Azure OpenAI\n", + "At the time of writing, access to Azure OpenAI service must be requested and approved. You can determine if you qualify for access and submit your request via [this form]( https://customervoice.microsoft.com/Pages/ResponsePage.aspx?id=v4j5cvGGr0GRqy180BHbR7en2Ais5pxKtso_Pz4b1_xUNTZBNzRKNlVQSFhZMU9aV09EVzYxWFdORCQlQCN0PWcu). Once access is granted, you must create an Azure OpenAI Service resource and deploy your model. [This tutorial](https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/create-resource?pivots=web-portal) walks through setting up an Azure OpenAI Service and deploying a model in a few simple steps. For this example, I’ve used GPT-3.5-Turbo, but not all models are available in all regions. Next, I recommend walking through [this tutorial](https://learn.microsoft.com/en-us/azure/ai-services/openai/quickstart?pivots=rest-api&tabs=command-line%2Cpython) with REST as your preferred method. Importantly, save your endpoint, API-key, and deployment name. We will need them for the next steps. Run the cell below and you can save each of them via the input boxes that appear. " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ec2aebc5-a4c6-4048-84cc-b0d9b15e9ed4", + "metadata": {}, + "outputs": [], + "source": [ + "import getpass\n", + "\n", + "endpoint = input(\"Endpoint: \")\n", + "deployment_name = input(\"Deployment name: \")\n", + "api_key = getpass.getpass(\"API Key: \")" + ] + }, + { + "cell_type": "markdown", + "id": "89abd7f2-200a-4b90-a029-d87513544775", + "metadata": {}, + "source": [ + "***\n", + "## Considerations for Key Management\n", + "Even though the cell above collects your API key, it uses the GetPass package to hide your key from view. With our endpoint, deployment name, and API key, anyone can make calls to our deployed LLM. Each call incurs a cost, so we need to prevent unauthorized usage. We want to take steps to protect our API key from falling into the wrong hands, but we need to use our API key in our score code to call our deployed LLM. Let’s outline a few key management options and discuss the potential risks of each. \n", + "\n", + "First, we can hardcode our key into the score code used in SAS Model Manager and SAS Intelligent Decisioning. This will give anyone with access to our models or Python code files on SAS Viya the ability to view our key, and thus, use our model. We can restrict access to these resources in SAS Viya by leveraging the [access control rules](https://go.documentation.sas.com/doc/en/sasadmincdc/v_049/evfun/n1uw3er96phzpfn1pxvnf01f6sw3.htm) built into SAS Viya so only a select few can see our models or Python code. \n", + "\n", + "We can make this approach a bit better by encoding our key using the Base64 package and then adding logic into our score code to decode our key prior to make the REST API call. Anyone can decode our encoded key, but now our key is no longer stored plain text. You can encode your key via:" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "9e99e596-b923-4c18-9a7f-6ba1132f86fb", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'ZXhhbXBsZV9rZXk='" + ] + }, + "execution_count": 9, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "import base64\n", + "key = 'example_key'\n", + "en = base64.b64encode(key.encode('ascii')).decode('ascii')\n", + "en" + ] + }, + { + "cell_type": "markdown", + "id": "6c645cd4-06a6-4637-bdf8-0acebba79954", + "metadata": {}, + "source": [ + "And you can decode your key by: " + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "id": "bbf669bd-473d-4f0b-b5f8-e3543abd8aa7", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'example_key'" + ] + }, + "execution_count": 10, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "de = base64.b64decode(en.encode('ascii')).decode('ascii')\n", + "de" + ] + }, + { + "cell_type": "markdown", + "id": "978fd715-96b1-496c-8222-c30e862edcb4", + "metadata": {}, + "source": [ + "Anyone with access to our model, decision, or deployment can use it to make a call to our deployed LLM. If we want to restrict usage to only folks with our key, we can add our key or our encoded key as an input to the model or decision. When calling the deployed model or decision, the individual must also pass the key as an input variable. When performing score tests in CAS or publishing validation, this will write the key to the CAS table, where it may be viewable to others. Additionally, when scoring via MAS, this may write the key to the log viewable by administrators. \n", + "\n", + "Finally, with help from an administrator, you can save the keys as Environment variables Python environment within SAS Viya. This allows anyone with access to the model, decision, or deployment access to our deployed LLM, but this does hide our key from others. For example, in a Python environment where the model to be executed, we can save an environment variable like so: " + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "id": "6ddd8878-ebe6-43ee-b088-035334e8dba3", + "metadata": {}, + "outputs": [], + "source": [ + "import os\n", + "os.environ['my_gpt_key'] = key" + ] + }, + { + "cell_type": "markdown", + "id": "f2c70757-bec8-49d9-ba7b-9e216c3da0f3", + "metadata": {}, + "source": [ + "And use it our code, like so: " + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "id": "f6b38665-2dba-49a3-98e9-535135e5c40c", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'example_key'" + ] + }, + "execution_count": 12, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "k = os.environ['my_gpt_key']\n", + "k" + ] + }, + { + "cell_type": "markdown", + "id": "f59bb4f6-936e-4ebd-86de-a0fb1bdb8183", + "metadata": {}, + "source": [ + "The key management techniques above can be combined to select the best level of security for your needs. You will need to update the code blocks below to take advantage of the techniques you’ve chosen. \n", + "*** \n", + "## Integration with SAS Model Manager\n", + "To run our model in SAS Model Manager, we need to write our score code and save it as a .py file. We also need to specify the inputs to our score code, the outputs the score code generates, and properties of the model. Then we can register it all into SAS Model Manager directly from this notebook using the python-sasctl package. First, let’s use python-sasctl to authenticate to our target SAS Viya environment. Run the cell below to input your information to create a session with SAS Viya. " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "8366fa36-4b33-4b1b-9729-dba501cecd51", + "metadata": {}, + "outputs": [], + "source": [ + "from sasctl import Session\n", + "from sasctl.services import model_repository as mr, model_management as mm\n", + "\n", + "host = input(\"Hostname: \")\n", + "username = input(\"Username: \")\n", + "password = getpass.getpass(\"Password: \")\n", + "\n", + "sess = Session(host, username, password)\n", + "sess" + ] + }, + { + "cell_type": "markdown", + "id": "4dcbf808-b3d1-430f-88c5-92bc65cbd83b", + "metadata": {}, + "source": [ + "Next, let’s write our score code. Keep in mind your chosen key management techniques and edit the score code to reflect your choice. In the block below, you need to: \n", + "1. Pick a name for your model and update the write file line to match your chosen name. This will write the file to your current directory, but you can edit this line to save the file to another directory. \n", + "1. Update the endpoint and deployment name within the URL string.\n", + "1. Update the key variable to reflect your key management strategy as well as make any necessary changes within SAS Viya.\n", + "\n", + "Run the block to write the score code file. " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "d0563c76-91a3-4c85-87f2-d99e13144899", + "metadata": {}, + "outputs": [], + "source": [ + "%%writefile [INSERT-YOUR-MODEL-NAME].py\n", + "import requests\n", + "\n", + "def score(prompt):\n", + " \"Output: answer_llm,finish_reason\"\n", + " \n", + " url = '[INSERT-YOUR-ENDPOINT]/openai/deployments/[INSERT-YOUR-DEPLOYMENT-NAME]/chat/completions?api-version=2023-05-15'\n", + "\n", + " k = '[INSERT-YOUR-API-KEY]'\n", + "\t\n", + " h = {\"Accept\": \"application/json\", \"Content-type\": \"application/json; charset=utf-8\", \"api-key\": k}\n", + "\n", + " data = {\"messages\":[{\"role\": \"user\", \"content\": prompt}]}\n", + "\n", + " response = requests.post(url, json = data , headers = h )\n", + "\n", + " jsonResponse = response.json() \n", + " finish_reason = jsonResponse[\"choices\"][0]['finish_reason']\n", + " answer_llm = jsonResponse[\"choices\"][0]['message']['content']\n", + "\n", + "\n", + " return answer_llm,finish_reason" + ] + }, + { + "cell_type": "markdown", + "id": "d90363ad-b7ec-4195-97e9-e56f1e3cf92f", + "metadata": {}, + "source": [ + "Next, let’s add our file to SAS Model Manager and update its properties. In the blow below, you need to:\n", + "1. Add the name of your project in SAS Model Manager, your chosen name for the model, and the algorithm from your deployed LLM, such as GPT 3-5 Turbo, in the first three lines of code.\n", + "1. Update the add_model_content function with your model name in the two indicated locations. \n", + "Once the code block has run, you can open your project in SAS Model Manager to find your model! From SAS Model Manager, you can run the model in a score test or deploy it to other destinations, including a container. " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c66efa86-6e80-4f35-aee4-ebd92c55c725", + "metadata": {}, + "outputs": [], + "source": [ + "# Update these variable to match your project\n", + "project = '[INSERT-YOUR-PROJECT-NAME]'\n", + "model_name = '[INSERT-YOUR-MODEL-NAME]'\n", + "algorithm = '[INSERT-YOUR-LLM-ALGORITHM]'\n", + "\n", + "# Specify input variables and output variables\n", + "inputvariables = [{'name': 'prompt', 'role': 'input', 'type': 'string', 'level': 'nominal', 'length': 500}]\n", + "outputvariables = [{'name': 'answer_llm', 'role': 'output', 'type': 'string', 'level': 'nominal', 'length': 500}, {'name': 'finish_reason', 'role': 'output', 'type': 'string', 'level': 'nominal', 'length': 15}]\n", + "\n", + "# Create the model\n", + "model = mr.create_model(\n", + " model=model_name,\n", + " project=project,\n", + " algorithm=algorithm,\n", + " modeler=username,\n", + " tool='Python 3',\n", + " function = \"Text Generation\",\n", + " score_code_type = 'Python',\n", + " input_variables = inputvariables,\n", + " output_variables = outputvariables\n", + ")\n", + "\n", + "# Add score code\n", + "scorefile = mr.add_model_content(\n", + " model,\n", + " open('[INSERT-YOUR-MODEL-NAME].py', 'rb'),\n", + " name='[INSERT-YOUR-MODEL-NAME].py',\n", + " role='score'\n", + ")\n" + ] + }, + { + "cell_type": "markdown", + "id": "0c758627-6995-4184-af93-966a8c6eacb2", + "metadata": {}, + "source": [ + "***\n", + "## Integration with SAS Intelligent Decisioning \n", + "We can use the Python model we just developed and registered to SAS Model Manager within our decision flow as a model. But, if you don’t have SAS Model Manager, no need to worry! We can leverage a Python code file in SAS Intelligent Decisioning instead. To run the GPT-3.5 model, we need to create an execute function that we will then copy and paste the code into the Python code files in SAS Intelligent Decisioning. The same concerns apply for key management, so update the code block below to reflect your key management strategy. In the code block below, you need to: \n", + "1. Update the endpoint and deployment name within the URL string. \n", + "1. Update the key variable to reflect your key management strategy as well as make any necessary changes within SAS Viya. \n", + "1. Copy the code block into a Python code file in SAS Intelligent Decisioning. " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a13e819d-4ba8-4174-b7ed-f9df56af8e1b", + "metadata": {}, + "outputs": [], + "source": [ + "''' List all output parameters as comma-separated values in the \"Output:\" docString. Do not specify \"None\" if there is no output parameter. '''\n", + "''' List all Python packages that are not built-in packages in the \"DependentPackages:\" docString. Separate the package names with commas on a single line. '''\n", + "\n", + "\n", + "import requests\n", + "\n", + "def execute (prompt):\n", + " 'Output:answer_llm,finish_reason'\n", + " 'DependentPackages: requests'\n", + " \n", + " url = '[INSERT-YOUR-ENDPOINT]/openai/deployments/[INSERT-YOUR-DEPLOYMENT-NAME]/chat/completions?api-version=2023-05-15'\n", + "\n", + " k = '[INSERT-YOUR-API-KEY]'\n", + "\t\n", + " h = {\"Accept\": \"application/json\", \"Content-type\": \"application/json; charset=utf-8\", \"api-key\": k}\n", + "\n", + " data = {\"messages\":[{\"role\": \"user\", \"content\": prompt}]}\n", + "\n", + " response = requests.post(url, json = data , headers = h )\n", + "\n", + " jsonResponse = response.json() \n", + " finish_reason = jsonResponse[\"choices\"][0]['finish_reason']\n", + " answer_llm = jsonResponse[\"choices\"][0]['message']['content']\n", + "\n", + "\n", + " return answer_llm,finish_reason\n" + ] + }, + { + "cell_type": "markdown", + "id": "6640cc88-94ac-4cb8-926c-715045635c08", + "metadata": {}, + "source": [ + "Once you add those changes to your code block above, it becomes a valid Python function and you can run here using the execute function to test it, like so: " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "06b09883-9c08-4647-a7f3-3de062fcf212", + "metadata": {}, + "outputs": [], + "source": [ + "execute(\"Write a tagline for an ice cream shop.\")" + ] + }, + { + "cell_type": "markdown", + "id": "bd0be03a-3765-4649-8a69-9decaee5a55b", + "metadata": {}, + "source": [ + "*** \n", + "## Conclusion\n", + "Now, you are all set to leverage a GPT-3.5-Turbo model deployed in Azure OpenAI in SAS Model Manager or SAS Intelligent Decisioning where it can be managed with other models in your organizations, combined with business logic, orchestrated with other models, and deployed into destinations within SAS Viya or beyond using containers." + ] + }, + { + "cell_type": "markdown", + "id": "31afee6d-b46b-4560-9c8a-33bcb41f2e93", + "metadata": {}, + "source": [ + "***" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.5" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} From d129ef0dfbcd9e66194503c1ae232d6b60fa450f Mon Sep 17 00:00:00 2001 From: Sophia Rowland Date: Thu, 27 Jun 2024 13:20:23 -0400 Subject: [PATCH 2/5] Update register_Azure_Open_AI_REST_Call.ipynb --- examples/register_Azure_Open_AI_REST_Call.ipynb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/examples/register_Azure_Open_AI_REST_Call.ipynb b/examples/register_Azure_Open_AI_REST_Call.ipynb index 5d99c16c..a1480e6b 100644 --- a/examples/register_Azure_Open_AI_REST_Call.ipynb +++ b/examples/register_Azure_Open_AI_REST_Call.ipynb @@ -6,7 +6,7 @@ "metadata": {}, "source": [ "Copyright © 2024, SAS Institute Inc., Cary, NC, USA. All Rights Reserved. SPDX-License-Identifier: Apache-2.0\n", - "# Using Azure OpenAI GPT Models in SAS Viya\n", + "# Register Azure OpenAI GPT Model REST API Calls\n", "In this notebook, we will walk through the steps required for leveraging a GPT-3.5-Turbo model from Azure OpenAI in SAS® Model Manager® and SAS® Intelligent Decisioning®. \n", "*** \n", "## Table of Contents \n", From af8852ec51d3df3a409813041ce29814d67bbf01 Mon Sep 17 00:00:00 2001 From: Sophia Rowland Date: Thu, 27 Jun 2024 13:23:27 -0400 Subject: [PATCH 3/5] Update README.md Updating Readme for new notebook --- examples/README.md | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/examples/README.md b/examples/README.md index 48b179d3..83832de7 100644 --- a/examples/README.md +++ b/examples/README.md @@ -37,6 +37,8 @@ Tasks and Services - [Modeling with Python & SAS AutoML](#modeling-with-python--sas-automl) - [Making direct REST API calls](#making-direct-rest-api-calls) +- [Register Azure OpenAI GPT Model REST API Calls](#register-Azure-OpenAI-GPT-Model-REST-API-Calls) + --- Register binary classification models ------------------------------------- @@ -206,3 +208,12 @@ Level: Advanced Demonstrates using `sasctl` to make REST calls over HTTP(S) directly to the SAS microservices. Use if you need to customize behavior or use functionality not yet exposed through higher-level `sasctl` functions. + + +Register Azure OpenAI GPT Model REST API Calls +-------------------------- +Filename: [register_Azure_Open_AI_REST_Call.ipynb](register_Azure_Open_AI_REST_Call.ipynb) + +Level: Intermediate + +Leverages a GPT-3.5-Turbo model from Azure OpenAI in SAS® Model Manager® and SAS® Intelligent Decisioning®. From e11764378e72f9d4fd82ab9c0bf498c79f4ef6d0 Mon Sep 17 00:00:00 2001 From: Kristen Aponte Date: Thu, 27 Jun 2024 18:45:32 -0400 Subject: [PATCH 4/5] Edited new Azure OpenAI GPT model example. --- examples/README.md | 11 ++-- ...Azure_OpenAI_model_using_REST_calls.ipynb} | 61 +++++++++---------- 2 files changed, 36 insertions(+), 36 deletions(-) rename examples/{register_Azure_Open_AI_REST_Call.ipynb => register_Azure_OpenAI_model_using_REST_calls.ipynb} (57%) diff --git a/examples/README.md b/examples/README.md index 83832de7..f91fe482 100644 --- a/examples/README.md +++ b/examples/README.md @@ -37,7 +37,9 @@ Tasks and Services - [Modeling with Python & SAS AutoML](#modeling-with-python--sas-automl) - [Making direct REST API calls](#making-direct-rest-api-calls) -- [Register Azure OpenAI GPT Model REST API Calls](#register-Azure-OpenAI-GPT-Model-REST-API-Calls) +- [Register an Azure OpenAI GPT model using REST API calls](#register-Azure-OpenAI-GPT-Model-REST-API-Calls) + +- Using Azure OpenAI GPT Models --- Register binary classification models @@ -209,11 +211,10 @@ Demonstrates using `sasctl` to make REST calls over HTTP(S) directly to the SAS Use if you need to customize behavior or use functionality not yet exposed through higher-level `sasctl` functions. - -Register Azure OpenAI GPT Model REST API Calls +Register an Azure OpenAI GPT Model Using REST API Calls -------------------------- -Filename: [register_Azure_Open_AI_REST_Call.ipynb](register_Azure_Open_AI_REST_Call.ipynb) +Filename: [register_Azure_OpenAI_model_using_REST_calls.ipynb](register_Azure_OpenAI_model_using_REST_calls.ipynb) Level: Intermediate -Leverages a GPT-3.5-Turbo model from Azure OpenAI in SAS® Model Manager® and SAS® Intelligent Decisioning®. +Leverages a GPT-3.5-Turbo model from Azure OpenAI in SAS® Model Manager and SAS® Intelligent Decisioning. diff --git a/examples/register_Azure_Open_AI_REST_Call.ipynb b/examples/register_Azure_OpenAI_model_using_REST_calls.ipynb similarity index 57% rename from examples/register_Azure_Open_AI_REST_Call.ipynb rename to examples/register_Azure_OpenAI_model_using_REST_calls.ipynb index a1480e6b..637ded8b 100644 --- a/examples/register_Azure_Open_AI_REST_Call.ipynb +++ b/examples/register_Azure_OpenAI_model_using_REST_calls.ipynb @@ -6,8 +6,8 @@ "metadata": {}, "source": [ "Copyright © 2024, SAS Institute Inc., Cary, NC, USA. All Rights Reserved. SPDX-License-Identifier: Apache-2.0\n", - "# Register Azure OpenAI GPT Model REST API Calls\n", - "In this notebook, we will walk through the steps required for leveraging a GPT-3.5-Turbo model from Azure OpenAI in SAS® Model Manager® and SAS® Intelligent Decisioning®. \n", + "# Register Azure OpenAI GPT Model Using REST API Calls\n", + "This notebook example, walks you through the steps required for leveraging a GPT-3.5-Turbo model from Azure OpenAI in SAS® Model Manager and SAS® Intelligent Decisioning. \n", "*** \n", "## Table of Contents \n", "1. [Introduction](#Introduction)\n", @@ -18,10 +18,10 @@ "1. [Conclusion](#Conclusion)\n", "***\n", "## Introduction \n", - "[Azure OpenAI Service](https://azure.microsoft.com/en-us/products/ai-services/openai-service/) provides access to Large Language Models (LLMs) through REST API, their Python SDK, and a web-based interface in the Azure OpenAI studio. Various models are available out of the box, including GPT-4, GPT-4 Turbo with Vision, GPT-3.5-Turbo, and Embeddings model series. You can read more about Azure OpenAI Service [here](https://learn.microsoft.com/en-us/azure/ai-services/openai/overview). To incorporate these models in SAS Intelligent Decisioning and SAS Model Manager, we will create a Python function and scoring code calling the REST API of a GPT-3.5-Turbo model deployed in Azure OpenAI. At the time of writing this notebook, the Python SDK does not support all calls to each model type, but that may change in the future. Additionally, we will walk through the steps required for deploying the model, based on what is required at the time of writing. \n", + "[Azure OpenAI Service](https://azure.microsoft.com/en-us/products/ai-services/openai-service/) provides access to Large Language Models (LLMs) through REST API, their Python SDK, and a web-based interface in the Azure OpenAI studio. Various models are available out of the box, including GPT-4, GPT-4 Turbo with Vision, GPT-3.5-Turbo, and Embeddings model series. For more information about the Azure OpenAI Service, [this article](https://learn.microsoft.com/en-us/azure/ai-services/openai/overview). To incorporate these models in SAS Intelligent Decisioning and SAS Model Manager, you create a Python function and score code by calling the REST API of a GPT-3.5-Turbo model deployed in Azure OpenAI. At the time of writing this notebook, the Python SDK does not support all calls to each model type, but that might change in the future. This example also walks you through the steps that are required for deploying the model, based on what was required when this example was created. \n", "*** \n", "## Deploying a GPT Model in Azure OpenAI\n", - "At the time of writing, access to Azure OpenAI service must be requested and approved. You can determine if you qualify for access and submit your request via [this form]( https://customervoice.microsoft.com/Pages/ResponsePage.aspx?id=v4j5cvGGr0GRqy180BHbR7en2Ais5pxKtso_Pz4b1_xUNTZBNzRKNlVQSFhZMU9aV09EVzYxWFdORCQlQCN0PWcu). Once access is granted, you must create an Azure OpenAI Service resource and deploy your model. [This tutorial](https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/create-resource?pivots=web-portal) walks through setting up an Azure OpenAI Service and deploying a model in a few simple steps. For this example, I’ve used GPT-3.5-Turbo, but not all models are available in all regions. Next, I recommend walking through [this tutorial](https://learn.microsoft.com/en-us/azure/ai-services/openai/quickstart?pivots=rest-api&tabs=command-line%2Cpython) with REST as your preferred method. Importantly, save your endpoint, API-key, and deployment name. We will need them for the next steps. Run the cell below and you can save each of them via the input boxes that appear. " + "Access to Azure OpenAI service must be requested and approved. You can determine whether you qualify for access and submit your request using [this form]( https://customervoice.microsoft.com/Pages/ResponsePage.aspx?id=v4j5cvGGr0GRqy180BHbR7en2Ais5pxKtso_Pz4b1_xUNTZBNzRKNlVQSFhZMU9aV09EVzYxWFdORCQlQCN0PWcu). After access is granted, you must create an Azure OpenAI Service resource and deploy your model. [This tutorial](https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/create-resource?pivots=web-portal) walks you through setting up an Azure OpenAI Service and deploying a model in a few simple steps. GPT-3.5-Turbo is used for this example, but not all models are available in all regions. Next, it is recommended that you walk through [this tutorial](https://learn.microsoft.com/en-us/azure/ai-services/openai/quickstart?pivots=rest-api&tabs=command-line%2Cpython) with REST as your preferred method. Make sure that you save your endpoint, API-key, and deployment name, because they are needed for the next steps. Run the cell below and then save each of them using the input boxes that appear. " ] }, { @@ -45,11 +45,11 @@ "source": [ "***\n", "## Considerations for Key Management\n", - "Even though the cell above collects your API key, it uses the GetPass package to hide your key from view. With our endpoint, deployment name, and API key, anyone can make calls to our deployed LLM. Each call incurs a cost, so we need to prevent unauthorized usage. We want to take steps to protect our API key from falling into the wrong hands, but we need to use our API key in our score code to call our deployed LLM. Let’s outline a few key management options and discuss the potential risks of each. \n", + "Even though the cell above collects your API key, it uses the GetPass package to hide your key from view. With the endpoint, deployment name, and API key, anyone can make calls to the deployed LLM. Each call incurs a cost, so you need to prevent unauthorized usage. You should take steps to protect the API key from falling into the wrong hands, but you must use the API key in the score code to call the deployed LLM. Here are a few key management options and the potential risks of each. \n", "\n", - "First, we can hardcode our key into the score code used in SAS Model Manager and SAS Intelligent Decisioning. This will give anyone with access to our models or Python code files on SAS Viya the ability to view our key, and thus, use our model. We can restrict access to these resources in SAS Viya by leveraging the [access control rules](https://go.documentation.sas.com/doc/en/sasadmincdc/v_049/evfun/n1uw3er96phzpfn1pxvnf01f6sw3.htm) built into SAS Viya so only a select few can see our models or Python code. \n", + "First, you can include the key in the score code used in SAS Model Manager and SAS Intelligent Decisioning. This gives anyone with access to the models or Python code files in SAS Viya the ability to view the key, and thus, use the model. You can restrict access to these resources in SAS Viya by leveraging the [access control rules](https://documentation.sas.com/?cdcId=sasadmincdc&cdcVersion=default&docsetId=evfun&docsetTarget=n1uw3er96phzpfn1pxvnf01f6sw3.htm) that are built into SAS Viya so that only a select few can view the models or Python code. \n", "\n", - "We can make this approach a bit better by encoding our key using the Base64 package and then adding logic into our score code to decode our key prior to make the REST API call. Anyone can decode our encoded key, but now our key is no longer stored plain text. You can encode your key via:" + "This approach can be made a bit better by encoding the key using the Base64 package and then adding logic into the score code to decode the key prior to making the REST API call. Anyone can decode the encoded key, but now the key is no longer stored as plain text. You can encode your key using this code:" ] }, { @@ -81,7 +81,7 @@ "id": "6c645cd4-06a6-4637-bdf8-0acebba79954", "metadata": {}, "source": [ - "And you can decode your key by: " + "And you can decode your key by using this code: " ] }, { @@ -111,9 +111,9 @@ "id": "978fd715-96b1-496c-8222-c30e862edcb4", "metadata": {}, "source": [ - "Anyone with access to our model, decision, or deployment can use it to make a call to our deployed LLM. If we want to restrict usage to only folks with our key, we can add our key or our encoded key as an input to the model or decision. When calling the deployed model or decision, the individual must also pass the key as an input variable. When performing score tests in CAS or publishing validation, this will write the key to the CAS table, where it may be viewable to others. Additionally, when scoring via MAS, this may write the key to the log viewable by administrators. \n", + "Anyone with access to the model, decision, or deployment can use it to make a call to the deployed LLM. If you want to restrict usage to only individuals with the key, you can add the key or the encoded key as an input to a model or decision. When calling the deployed model or decision, the individual must also pass the key as an input variable. When performing score tests or publishing validation tests in CAS, this writes the key to the CAS table, where it can be viewable to others. Also, when scoring using the SAS Micro Analytic Service, this can write the key to the log viewable by administrators. \n", "\n", - "Finally, with help from an administrator, you can save the keys as Environment variables Python environment within SAS Viya. This allows anyone with access to the model, decision, or deployment access to our deployed LLM, but this does hide our key from others. For example, in a Python environment where the model to be executed, we can save an environment variable like so: " + "Finally, with help from an administrator, you can save the keys as environment variables in the Python environment within SAS Viya. This enables anyone with access to the model, decision, or deployment access to the deployed LLM, but this does hide the key from others. For example, in a Python environment where the model is executed, you can save an environment variable in this way: " ] }, { @@ -132,7 +132,7 @@ "id": "f2c70757-bec8-49d9-ba7b-9e216c3da0f3", "metadata": {}, "source": [ - "And use it our code, like so: " + "And use the code, like so: " ] }, { @@ -162,10 +162,10 @@ "id": "f59bb4f6-936e-4ebd-86de-a0fb1bdb8183", "metadata": {}, "source": [ - "The key management techniques above can be combined to select the best level of security for your needs. You will need to update the code blocks below to take advantage of the techniques you’ve chosen. \n", + "The key management techniques above can be combined to select the best level of security for your needs. You must update the code blocks below to take advantage of the techniques that you have chosen. \n", "*** \n", "## Integration with SAS Model Manager\n", - "To run our model in SAS Model Manager, we need to write our score code and save it as a .py file. We also need to specify the inputs to our score code, the outputs the score code generates, and properties of the model. Then we can register it all into SAS Model Manager directly from this notebook using the python-sasctl package. First, let’s use python-sasctl to authenticate to our target SAS Viya environment. Run the cell below to input your information to create a session with SAS Viya. " + "To run the model in SAS Model Manager, you must write the score code and save it as a Python (.py) file. You also must specify the inputs in the score code. The score code generates the outputs, and properties of the model. You can then register it all into SAS Model Manager directly from this notebook using the python-sasctl package. First, use python-sasctl to authenticate with the target SAS Viya environment. Run the cell below to specify your information to create a session with SAS Viya. " ] }, { @@ -178,8 +178,8 @@ "from sasctl import Session\n", "from sasctl.services import model_repository as mr, model_management as mm\n", "\n", - "host = input(\"Hostname: \")\n", - "username = input(\"Username: \")\n", + "host = input(\"Host name: \")\n", + "username = input(\"User name: \")\n", "password = getpass.getpass(\"Password: \")\n", "\n", "sess = Session(host, username, password)\n", @@ -191,12 +191,11 @@ "id": "4dcbf808-b3d1-430f-88c5-92bc65cbd83b", "metadata": {}, "source": [ - "Next, let’s write our score code. Keep in mind your chosen key management techniques and edit the score code to reflect your choice. In the block below, you need to: \n", - "1. Pick a name for your model and update the write file line to match your chosen name. This will write the file to your current directory, but you can edit this line to save the file to another directory. \n", - "1. Update the endpoint and deployment name within the URL string.\n", - "1. Update the key variable to reflect your key management strategy as well as make any necessary changes within SAS Viya.\n", - "\n", - "Run the block to write the score code file. " + "Next, write the score code. Keep in mind your chosen key management techniques and edit the score code to reflect your choice. In the code block below, you must perform these steps: \n", + "1. Choose a name for your model and update the `writefile` line with that name. This writes the file to your current directory, but you can edit this line to save the file to another directory. \n", + "2. Update the endpoint and deployment name within the URL string.\n", + "3. Update the key variable to reflect your key management strategy as well as make any necessary changes within SAS Viya.\n", + "4. Run the block to write the score code file. " ] }, { @@ -235,10 +234,10 @@ "id": "d90363ad-b7ec-4195-97e9-e56f1e3cf92f", "metadata": {}, "source": [ - "Next, let’s add our file to SAS Model Manager and update its properties. In the blow below, you need to:\n", - "1. Add the name of your project in SAS Model Manager, your chosen name for the model, and the algorithm from your deployed LLM, such as GPT 3-5 Turbo, in the first three lines of code.\n", - "1. Update the add_model_content function with your model name in the two indicated locations. \n", - "Once the code block has run, you can open your project in SAS Model Manager to find your model! From SAS Model Manager, you can run the model in a score test or deploy it to other destinations, including a container. " + "Next, add the file to SAS Model Manager and update its properties. In the below code block, you must perform these steps:\n", + "1. In the first three lines of code, specify the name of your project in SAS Model Manager, the name of your model, and the algorithm from your deployed LLM, such as GPT 3.5 Turbo. \n", + "2. Update the add_model_content function with your model name in the two indicated locations. \n", + "After the code block has run, you can open your project in SAS Model Manager to find your model. Using SAS Model Manager, you can run score test for the model, or deploy it to other destinations, including a container. " ] }, { @@ -248,7 +247,7 @@ "metadata": {}, "outputs": [], "source": [ - "# Update these variable to match your project\n", + "# Update these variables to match your project\n", "project = '[INSERT-YOUR-PROJECT-NAME]'\n", "model_name = '[INSERT-YOUR-MODEL-NAME]'\n", "algorithm = '[INSERT-YOUR-LLM-ALGORITHM]'\n", @@ -286,10 +285,10 @@ "source": [ "***\n", "## Integration with SAS Intelligent Decisioning \n", - "We can use the Python model we just developed and registered to SAS Model Manager within our decision flow as a model. But, if you don’t have SAS Model Manager, no need to worry! We can leverage a Python code file in SAS Intelligent Decisioning instead. To run the GPT-3.5 model, we need to create an execute function that we will then copy and paste the code into the Python code files in SAS Intelligent Decisioning. The same concerns apply for key management, so update the code block below to reflect your key management strategy. In the code block below, you need to: \n", + "You can include the Python model that you just developed and registered in SAS Model Manager within a decision flow. But, if you do not have SAS Model Manager, no need to worry, you can leverage a Python code file in SAS Intelligent Decisioning instead. To run the GPT-3.5 model, you must create an execute function that you can then copy and paste into the Python code files in SAS Intelligent Decisioning. The same concerns apply to key management, so update the below code block to reflect your key management strategy. In the code block below, you must perform these steps: \n", "1. Update the endpoint and deployment name within the URL string. \n", - "1. Update the key variable to reflect your key management strategy as well as make any necessary changes within SAS Viya. \n", - "1. Copy the code block into a Python code file in SAS Intelligent Decisioning. " + "2. Update the key variable to reflect your key management strategy as well as make any necessary changes within SAS Viya. \n", + "3. Copy the code block into a Python code file in SAS Intelligent Decisioning. " ] }, { @@ -332,7 +331,7 @@ "id": "6640cc88-94ac-4cb8-926c-715045635c08", "metadata": {}, "source": [ - "Once you add those changes to your code block above, it becomes a valid Python function and you can run here using the execute function to test it, like so: " + "After you add those changes to the above code block, it becomes a valid Python function, and then you can run it using the execute function to test it. " ] }, { @@ -352,7 +351,7 @@ "source": [ "*** \n", "## Conclusion\n", - "Now, you are all set to leverage a GPT-3.5-Turbo model deployed in Azure OpenAI in SAS Model Manager or SAS Intelligent Decisioning where it can be managed with other models in your organizations, combined with business logic, orchestrated with other models, and deployed into destinations within SAS Viya or beyond using containers." + "Now, you are all set to leverage a GPT-3.5-Turbo model that is deployed in Azure OpenAI with SAS Model Manager or SAS Intelligent Decisioning. It can then be managed with other models within your organization, combined with business logic, orchestrated with other models, and deployed into destinations within SAS Viya or beyond using containers." ] }, { From 9c55803c65f337ec28ae48b58523d4a9910ddf85 Mon Sep 17 00:00:00 2001 From: Kristen Aponte Date: Thu, 27 Jun 2024 18:54:48 -0400 Subject: [PATCH 5/5] Edited examples README.md file. --- examples/README.md | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/examples/README.md b/examples/README.md index f91fe482..947d4f82 100644 --- a/examples/README.md +++ b/examples/README.md @@ -37,9 +37,7 @@ Tasks and Services - [Modeling with Python & SAS AutoML](#modeling-with-python--sas-automl) - [Making direct REST API calls](#making-direct-rest-api-calls) -- [Register an Azure OpenAI GPT model using REST API calls](#register-Azure-OpenAI-GPT-Model-REST-API-Calls) - -- Using Azure OpenAI GPT Models +- [Register an Azure OpenAI GPT model using REST API calls](#register-an-azure-openai-gpt-model-using-rest-api-calls) --- Register binary classification models @@ -98,11 +96,11 @@ Generates a requirements.json file which includes the minimal number of dependen Create and update custom model KPIs ----------------------------------- -Filename: [pzmm_custom_kpis.ipynb](pzmm_custom_kpis.ipynb) +Filename: [pzmm_custom_kpis.ipynb](pzmm_custom_kpi_model_parameters.ipynb) Level: Intermediate -Create and update custom model parameters and kpis on SAS Model Manager +Create and update custom model parameters and kpis on SAS Model Manager. Register a SAS classification model