diff --git a/examples/README.md b/examples/README.md index 48b179d3..947d4f82 100644 --- a/examples/README.md +++ b/examples/README.md @@ -37,6 +37,8 @@ Tasks and Services - [Modeling with Python & SAS AutoML](#modeling-with-python--sas-automl) - [Making direct REST API calls](#making-direct-rest-api-calls) +- [Register an Azure OpenAI GPT model using REST API calls](#register-an-azure-openai-gpt-model-using-rest-api-calls) + --- Register binary classification models ------------------------------------- @@ -94,11 +96,11 @@ Generates a requirements.json file which includes the minimal number of dependen Create and update custom model KPIs ----------------------------------- -Filename: [pzmm_custom_kpis.ipynb](pzmm_custom_kpis.ipynb) +Filename: [pzmm_custom_kpis.ipynb](pzmm_custom_kpi_model_parameters.ipynb) Level: Intermediate -Create and update custom model parameters and kpis on SAS Model Manager +Create and update custom model parameters and kpis on SAS Model Manager. Register a SAS classification model @@ -206,3 +208,11 @@ Level: Advanced Demonstrates using `sasctl` to make REST calls over HTTP(S) directly to the SAS microservices. Use if you need to customize behavior or use functionality not yet exposed through higher-level `sasctl` functions. + +Register an Azure OpenAI GPT Model Using REST API Calls +-------------------------- +Filename: [register_Azure_OpenAI_model_using_REST_calls.ipynb](register_Azure_OpenAI_model_using_REST_calls.ipynb) + +Level: Intermediate + +Leverages a GPT-3.5-Turbo model from Azure OpenAI in SAS® Model Manager and SAS® Intelligent Decisioning. diff --git a/examples/register_Azure_OpenAI_model_using_REST_calls.ipynb b/examples/register_Azure_OpenAI_model_using_REST_calls.ipynb new file mode 100644 index 00000000..637ded8b --- /dev/null +++ b/examples/register_Azure_OpenAI_model_using_REST_calls.ipynb @@ -0,0 +1,387 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "3e928632-2b60-4eea-b296-1b196a29b3a4", + "metadata": {}, + "source": [ + "Copyright © 2024, SAS Institute Inc., Cary, NC, USA. All Rights Reserved. SPDX-License-Identifier: Apache-2.0\n", + "# Register Azure OpenAI GPT Model Using REST API Calls\n", + "This notebook example, walks you through the steps required for leveraging a GPT-3.5-Turbo model from Azure OpenAI in SAS® Model Manager and SAS® Intelligent Decisioning. \n", + "*** \n", + "## Table of Contents \n", + "1. [Introduction](#Introduction)\n", + "1. [Deploying a GPT Model in Azure OpenAI](#Deploying-a-GPT-Model-in-Azure-OpenAI)\n", + "1. [Considerations for Key Management](#Considerations-for-Key-Management)\n", + "1. [Integration with SAS Model Manager](#Integration-with-SAS-Model-Manager)\n", + "1. [Integration with SAS Intelligent Decisioning](#Integration-with-SAS-Intelligent-Decisioning)\n", + "1. [Conclusion](#Conclusion)\n", + "***\n", + "## Introduction \n", + "[Azure OpenAI Service](https://azure.microsoft.com/en-us/products/ai-services/openai-service/) provides access to Large Language Models (LLMs) through REST API, their Python SDK, and a web-based interface in the Azure OpenAI studio. Various models are available out of the box, including GPT-4, GPT-4 Turbo with Vision, GPT-3.5-Turbo, and Embeddings model series. For more information about the Azure OpenAI Service, [this article](https://learn.microsoft.com/en-us/azure/ai-services/openai/overview). To incorporate these models in SAS Intelligent Decisioning and SAS Model Manager, you create a Python function and score code by calling the REST API of a GPT-3.5-Turbo model deployed in Azure OpenAI. At the time of writing this notebook, the Python SDK does not support all calls to each model type, but that might change in the future. This example also walks you through the steps that are required for deploying the model, based on what was required when this example was created. \n", + "*** \n", + "## Deploying a GPT Model in Azure OpenAI\n", + "Access to Azure OpenAI service must be requested and approved. You can determine whether you qualify for access and submit your request using [this form]( https://customervoice.microsoft.com/Pages/ResponsePage.aspx?id=v4j5cvGGr0GRqy180BHbR7en2Ais5pxKtso_Pz4b1_xUNTZBNzRKNlVQSFhZMU9aV09EVzYxWFdORCQlQCN0PWcu). After access is granted, you must create an Azure OpenAI Service resource and deploy your model. [This tutorial](https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/create-resource?pivots=web-portal) walks you through setting up an Azure OpenAI Service and deploying a model in a few simple steps. GPT-3.5-Turbo is used for this example, but not all models are available in all regions. Next, it is recommended that you walk through [this tutorial](https://learn.microsoft.com/en-us/azure/ai-services/openai/quickstart?pivots=rest-api&tabs=command-line%2Cpython) with REST as your preferred method. Make sure that you save your endpoint, API-key, and deployment name, because they are needed for the next steps. Run the cell below and then save each of them using the input boxes that appear. " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ec2aebc5-a4c6-4048-84cc-b0d9b15e9ed4", + "metadata": {}, + "outputs": [], + "source": [ + "import getpass\n", + "\n", + "endpoint = input(\"Endpoint: \")\n", + "deployment_name = input(\"Deployment name: \")\n", + "api_key = getpass.getpass(\"API Key: \")" + ] + }, + { + "cell_type": "markdown", + "id": "89abd7f2-200a-4b90-a029-d87513544775", + "metadata": {}, + "source": [ + "***\n", + "## Considerations for Key Management\n", + "Even though the cell above collects your API key, it uses the GetPass package to hide your key from view. With the endpoint, deployment name, and API key, anyone can make calls to the deployed LLM. Each call incurs a cost, so you need to prevent unauthorized usage. You should take steps to protect the API key from falling into the wrong hands, but you must use the API key in the score code to call the deployed LLM. Here are a few key management options and the potential risks of each. \n", + "\n", + "First, you can include the key in the score code used in SAS Model Manager and SAS Intelligent Decisioning. This gives anyone with access to the models or Python code files in SAS Viya the ability to view the key, and thus, use the model. You can restrict access to these resources in SAS Viya by leveraging the [access control rules](https://documentation.sas.com/?cdcId=sasadmincdc&cdcVersion=default&docsetId=evfun&docsetTarget=n1uw3er96phzpfn1pxvnf01f6sw3.htm) that are built into SAS Viya so that only a select few can view the models or Python code. \n", + "\n", + "This approach can be made a bit better by encoding the key using the Base64 package and then adding logic into the score code to decode the key prior to making the REST API call. Anyone can decode the encoded key, but now the key is no longer stored as plain text. You can encode your key using this code:" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "9e99e596-b923-4c18-9a7f-6ba1132f86fb", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'ZXhhbXBsZV9rZXk='" + ] + }, + "execution_count": 9, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "import base64\n", + "key = 'example_key'\n", + "en = base64.b64encode(key.encode('ascii')).decode('ascii')\n", + "en" + ] + }, + { + "cell_type": "markdown", + "id": "6c645cd4-06a6-4637-bdf8-0acebba79954", + "metadata": {}, + "source": [ + "And you can decode your key by using this code: " + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "id": "bbf669bd-473d-4f0b-b5f8-e3543abd8aa7", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'example_key'" + ] + }, + "execution_count": 10, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "de = base64.b64decode(en.encode('ascii')).decode('ascii')\n", + "de" + ] + }, + { + "cell_type": "markdown", + "id": "978fd715-96b1-496c-8222-c30e862edcb4", + "metadata": {}, + "source": [ + "Anyone with access to the model, decision, or deployment can use it to make a call to the deployed LLM. If you want to restrict usage to only individuals with the key, you can add the key or the encoded key as an input to a model or decision. When calling the deployed model or decision, the individual must also pass the key as an input variable. When performing score tests or publishing validation tests in CAS, this writes the key to the CAS table, where it can be viewable to others. Also, when scoring using the SAS Micro Analytic Service, this can write the key to the log viewable by administrators. \n", + "\n", + "Finally, with help from an administrator, you can save the keys as environment variables in the Python environment within SAS Viya. This enables anyone with access to the model, decision, or deployment access to the deployed LLM, but this does hide the key from others. For example, in a Python environment where the model is executed, you can save an environment variable in this way: " + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "id": "6ddd8878-ebe6-43ee-b088-035334e8dba3", + "metadata": {}, + "outputs": [], + "source": [ + "import os\n", + "os.environ['my_gpt_key'] = key" + ] + }, + { + "cell_type": "markdown", + "id": "f2c70757-bec8-49d9-ba7b-9e216c3da0f3", + "metadata": {}, + "source": [ + "And use the code, like so: " + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "id": "f6b38665-2dba-49a3-98e9-535135e5c40c", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'example_key'" + ] + }, + "execution_count": 12, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "k = os.environ['my_gpt_key']\n", + "k" + ] + }, + { + "cell_type": "markdown", + "id": "f59bb4f6-936e-4ebd-86de-a0fb1bdb8183", + "metadata": {}, + "source": [ + "The key management techniques above can be combined to select the best level of security for your needs. You must update the code blocks below to take advantage of the techniques that you have chosen. \n", + "*** \n", + "## Integration with SAS Model Manager\n", + "To run the model in SAS Model Manager, you must write the score code and save it as a Python (.py) file. You also must specify the inputs in the score code. The score code generates the outputs, and properties of the model. You can then register it all into SAS Model Manager directly from this notebook using the python-sasctl package. First, use python-sasctl to authenticate with the target SAS Viya environment. Run the cell below to specify your information to create a session with SAS Viya. " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "8366fa36-4b33-4b1b-9729-dba501cecd51", + "metadata": {}, + "outputs": [], + "source": [ + "from sasctl import Session\n", + "from sasctl.services import model_repository as mr, model_management as mm\n", + "\n", + "host = input(\"Host name: \")\n", + "username = input(\"User name: \")\n", + "password = getpass.getpass(\"Password: \")\n", + "\n", + "sess = Session(host, username, password)\n", + "sess" + ] + }, + { + "cell_type": "markdown", + "id": "4dcbf808-b3d1-430f-88c5-92bc65cbd83b", + "metadata": {}, + "source": [ + "Next, write the score code. Keep in mind your chosen key management techniques and edit the score code to reflect your choice. In the code block below, you must perform these steps: \n", + "1. Choose a name for your model and update the `writefile` line with that name. This writes the file to your current directory, but you can edit this line to save the file to another directory. \n", + "2. Update the endpoint and deployment name within the URL string.\n", + "3. Update the key variable to reflect your key management strategy as well as make any necessary changes within SAS Viya.\n", + "4. Run the block to write the score code file. " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "d0563c76-91a3-4c85-87f2-d99e13144899", + "metadata": {}, + "outputs": [], + "source": [ + "%%writefile [INSERT-YOUR-MODEL-NAME].py\n", + "import requests\n", + "\n", + "def score(prompt):\n", + " \"Output: answer_llm,finish_reason\"\n", + " \n", + " url = '[INSERT-YOUR-ENDPOINT]/openai/deployments/[INSERT-YOUR-DEPLOYMENT-NAME]/chat/completions?api-version=2023-05-15'\n", + "\n", + " k = '[INSERT-YOUR-API-KEY]'\n", + "\t\n", + " h = {\"Accept\": \"application/json\", \"Content-type\": \"application/json; charset=utf-8\", \"api-key\": k}\n", + "\n", + " data = {\"messages\":[{\"role\": \"user\", \"content\": prompt}]}\n", + "\n", + " response = requests.post(url, json = data , headers = h )\n", + "\n", + " jsonResponse = response.json() \n", + " finish_reason = jsonResponse[\"choices\"][0]['finish_reason']\n", + " answer_llm = jsonResponse[\"choices\"][0]['message']['content']\n", + "\n", + "\n", + " return answer_llm,finish_reason" + ] + }, + { + "cell_type": "markdown", + "id": "d90363ad-b7ec-4195-97e9-e56f1e3cf92f", + "metadata": {}, + "source": [ + "Next, add the file to SAS Model Manager and update its properties. In the below code block, you must perform these steps:\n", + "1. In the first three lines of code, specify the name of your project in SAS Model Manager, the name of your model, and the algorithm from your deployed LLM, such as GPT 3.5 Turbo. \n", + "2. Update the add_model_content function with your model name in the two indicated locations. \n", + "After the code block has run, you can open your project in SAS Model Manager to find your model. Using SAS Model Manager, you can run score test for the model, or deploy it to other destinations, including a container. " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c66efa86-6e80-4f35-aee4-ebd92c55c725", + "metadata": {}, + "outputs": [], + "source": [ + "# Update these variables to match your project\n", + "project = '[INSERT-YOUR-PROJECT-NAME]'\n", + "model_name = '[INSERT-YOUR-MODEL-NAME]'\n", + "algorithm = '[INSERT-YOUR-LLM-ALGORITHM]'\n", + "\n", + "# Specify input variables and output variables\n", + "inputvariables = [{'name': 'prompt', 'role': 'input', 'type': 'string', 'level': 'nominal', 'length': 500}]\n", + "outputvariables = [{'name': 'answer_llm', 'role': 'output', 'type': 'string', 'level': 'nominal', 'length': 500}, {'name': 'finish_reason', 'role': 'output', 'type': 'string', 'level': 'nominal', 'length': 15}]\n", + "\n", + "# Create the model\n", + "model = mr.create_model(\n", + " model=model_name,\n", + " project=project,\n", + " algorithm=algorithm,\n", + " modeler=username,\n", + " tool='Python 3',\n", + " function = \"Text Generation\",\n", + " score_code_type = 'Python',\n", + " input_variables = inputvariables,\n", + " output_variables = outputvariables\n", + ")\n", + "\n", + "# Add score code\n", + "scorefile = mr.add_model_content(\n", + " model,\n", + " open('[INSERT-YOUR-MODEL-NAME].py', 'rb'),\n", + " name='[INSERT-YOUR-MODEL-NAME].py',\n", + " role='score'\n", + ")\n" + ] + }, + { + "cell_type": "markdown", + "id": "0c758627-6995-4184-af93-966a8c6eacb2", + "metadata": {}, + "source": [ + "***\n", + "## Integration with SAS Intelligent Decisioning \n", + "You can include the Python model that you just developed and registered in SAS Model Manager within a decision flow. But, if you do not have SAS Model Manager, no need to worry, you can leverage a Python code file in SAS Intelligent Decisioning instead. To run the GPT-3.5 model, you must create an execute function that you can then copy and paste into the Python code files in SAS Intelligent Decisioning. The same concerns apply to key management, so update the below code block to reflect your key management strategy. In the code block below, you must perform these steps: \n", + "1. Update the endpoint and deployment name within the URL string. \n", + "2. Update the key variable to reflect your key management strategy as well as make any necessary changes within SAS Viya. \n", + "3. Copy the code block into a Python code file in SAS Intelligent Decisioning. " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a13e819d-4ba8-4174-b7ed-f9df56af8e1b", + "metadata": {}, + "outputs": [], + "source": [ + "''' List all output parameters as comma-separated values in the \"Output:\" docString. Do not specify \"None\" if there is no output parameter. '''\n", + "''' List all Python packages that are not built-in packages in the \"DependentPackages:\" docString. Separate the package names with commas on a single line. '''\n", + "\n", + "\n", + "import requests\n", + "\n", + "def execute (prompt):\n", + " 'Output:answer_llm,finish_reason'\n", + " 'DependentPackages: requests'\n", + " \n", + " url = '[INSERT-YOUR-ENDPOINT]/openai/deployments/[INSERT-YOUR-DEPLOYMENT-NAME]/chat/completions?api-version=2023-05-15'\n", + "\n", + " k = '[INSERT-YOUR-API-KEY]'\n", + "\t\n", + " h = {\"Accept\": \"application/json\", \"Content-type\": \"application/json; charset=utf-8\", \"api-key\": k}\n", + "\n", + " data = {\"messages\":[{\"role\": \"user\", \"content\": prompt}]}\n", + "\n", + " response = requests.post(url, json = data , headers = h )\n", + "\n", + " jsonResponse = response.json() \n", + " finish_reason = jsonResponse[\"choices\"][0]['finish_reason']\n", + " answer_llm = jsonResponse[\"choices\"][0]['message']['content']\n", + "\n", + "\n", + " return answer_llm,finish_reason\n" + ] + }, + { + "cell_type": "markdown", + "id": "6640cc88-94ac-4cb8-926c-715045635c08", + "metadata": {}, + "source": [ + "After you add those changes to the above code block, it becomes a valid Python function, and then you can run it using the execute function to test it. " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "06b09883-9c08-4647-a7f3-3de062fcf212", + "metadata": {}, + "outputs": [], + "source": [ + "execute(\"Write a tagline for an ice cream shop.\")" + ] + }, + { + "cell_type": "markdown", + "id": "bd0be03a-3765-4649-8a69-9decaee5a55b", + "metadata": {}, + "source": [ + "*** \n", + "## Conclusion\n", + "Now, you are all set to leverage a GPT-3.5-Turbo model that is deployed in Azure OpenAI with SAS Model Manager or SAS Intelligent Decisioning. It can then be managed with other models within your organization, combined with business logic, orchestrated with other models, and deployed into destinations within SAS Viya or beyond using containers." + ] + }, + { + "cell_type": "markdown", + "id": "31afee6d-b46b-4560-9c8a-33bcb41f2e93", + "metadata": {}, + "source": [ + "***" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.5" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +}