L4_Production-ready_techniques.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "32d013f7",
   "metadata": {},
   "source": [
    "# L4: Production-ready techniques"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f0d3bbc6",
   "metadata": {},
   "source": [
    "In this lesson, you are going to learn about some production techniques, and integrate them into a web app."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "49412c5b",
   "metadata": {},
   "source": [
    "### Import required dependencies and load the API key"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "df8739ce-2780-4c8a-b52d-0fb81edaa365",
   "metadata": {
    "height": 183
   },
   "outputs": [],
   "source": [
    "import * as mod from \"https://deno.land/std@0.213.0/dotenv/mod.ts\";\n",
    "import { \n",
    "    Document, \n",
    "    VectorStoreIndex, \n",
    "    SimpleDirectoryReader,\n",
    "    RouterQueryEngine,\n",
    "    storageContextFromDefaults,\n",
    "    ContextChatEngine\n",
    "} from \"npm:llamaindex@0.1.12\"\n",
    "const keys = await mod.load({export:true}) // read API key from .env"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "99e39887",
   "metadata": {},
   "source": [
    "## 1. Persisting your data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1407d627",
   "metadata": {},
   "source": [
    "### Set up a storage context"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "382754f9-07a9-4a05-849c-184ab340c882",
   "metadata": {
    "height": 64
   },
   "outputs": [],
   "source": [
    "const storageContext = await storageContextFromDefaults({\n",
    "  persistDir: \"./storage\",\n",
    "});"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2de8fa27",
   "metadata": {},
   "source": [
    "### Load the data and create an index"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "0a83bc49-d1cf-43b8-b5f2-e229de0920ba",
   "metadata": {
    "height": 115
   },
   "outputs": [],
   "source": [
    "const documents = await new SimpleDirectoryReader().loadData({\n",
    "  directoryPath: \"./data\", // we have the React wikipedia page in here\n",
    "});\n",
    "let index = await VectorStoreIndex.fromDocuments(documents, {\n",
    "  storageContext\n",
    "});"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "70183617",
   "metadata": {},
   "source": [
    "### Let's ask it a question!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "afe86f98-219e-46ba-b577-39e29f8adbd0",
   "metadata": {
    "height": 64
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "JSX, or JavaScript Syntax Extension, is an extension to the JavaScript language syntax. It provides a way to structure component rendering using syntax familiar to many developers. React components are typically written using JSX, although they do not have to be (components may also be written in pure JavaScript).\n"
     ]
    }
   ],
   "source": [
    "let engine = await index.asQueryEngine()\n",
    "let response = await engine.query({query:\"What is JSX?\"})\n",
    "console.log(response.toString())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7607dfb3",
   "metadata": {},
   "source": [
    "### Get an index without parsing the documents "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "d2d47e4a-e6c0-415f-92be-45ebba67be98",
   "metadata": {
    "height": 64
   },
   "outputs": [],
   "source": [
    "let storageContext2 = await storageContextFromDefaults({\n",
    "    persistDir: \"./storage\",\n",
    "});"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a91dad78",
   "metadata": {},
   "source": [
    "\n",
    "### Initialize the index"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "8d6f0326-7144-4b5b-95f5-ba5f3e9cfc21",
   "metadata": {
    "height": 64
   },
   "outputs": [],
   "source": [
    "let index2 = await VectorStoreIndex.init({\n",
    "    storageContext: storageContext2\n",
    "});"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "23ab2471",
   "metadata": {},
   "source": [
    "### Let's query it!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "d154581a-2a5c-4143-ac49-c98107d8262f",
   "metadata": {
    "height": 64
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "JSX, or JavaScript Syntax Extension, is an extension to the JavaScript language syntax. It provides a way to structure component rendering using syntax familiar to many developers. React components are typically written using JSX, although they do not have to be. It is similar to another extension syntax created by Facebook for PHP called XHP.\n"
     ]
    }
   ],
   "source": [
    "let engine2 = await index2.asQueryEngine()\n",
    "let response2 = await engine2.query({query:\"What is JSX?\"})\n",
    "console.log(response2.toString())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "988aecd0",
   "metadata": {},
   "source": [
    "## 2. Chatting with your data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4bedb39c",
   "metadata": {},
   "source": [
    "### Create a retriever and a new chat engine"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "19489e8c-d71a-4b21-bccf-245cd8be42ae",
   "metadata": {
    "height": 132
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\u001b[33m3\u001b[39m"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "const retriever = index2.asRetriever();\n",
    "retriever.similarityTopK = 3;\n",
    "\n",
    "let chatEngine = new ContextChatEngine({\n",
    "    retriever,\n",
    "});\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8d0bc17d",
   "metadata": {},
   "source": [
    "### Let's try it!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "5eaebc17-aa7e-452b-ab89-493726fc66a2",
   "metadata": {
    "height": 202
   },
   "outputs": [],
   "source": [
    "let messageHistory = [\n",
    "    {\n",
    "        role: \"user\",\n",
    "        content: \"What is JSX?\"\n",
    "    },\n",
    "    {\n",
    "        role: \"assistant\",\n",
    "        content: \"JSX stands for JavaScript Syntax Extension. It is an extension to the JavaScript language syntax that provides a way to structure component rendering using syntax familiar to many developers. JSX is similar in appearance to HTML and is typically used to write React components, although components can also be written in pure JavaScript. It was created by Facebook and is similar to another extension syntax created for PHP called XHP.\"\n",
    "    }\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "bc0a2005-096e-44f2-9949-06ff3419a011",
   "metadata": {
    "height": 47
   },
   "outputs": [],
   "source": [
    "let newMessage = \"What was that last thing you mentioned?\"\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "1a18f788-f146-48b9-ba39-f224ad829327",
   "metadata": {
    "height": 98
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "I mentioned XHP, which stands for XML HTML Parser. XHP is an extension syntax for PHP that allows developers to write HTML and XML-like structures directly in their PHP code. It was also created by Facebook and provides a way to write structured and type-safe code for building web applications in PHP. XHP was an inspiration for JSX in the React ecosystem.\n"
     ]
    }
   ],
   "source": [
    "const response3 = await chatEngine.chat({\n",
    "  message: newMessage,\n",
    "  chatHistory: messageHistory\n",
    "});\n",
    "console.log(response3.toString())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4c466165",
   "metadata": {},
   "source": [
    "## 3. Streaming"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "70fcbe70",
   "metadata": {},
   "source": [
    "### Setting stream to true!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "67a85874-e775-4175-8936-0294020dbdd5",
   "metadata": {
    "height": 115
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_async_generator$4 { _invoke: [Function: send] }\n"
     ]
    }
   ],
   "source": [
    "const response4 = await chatEngine.chat({\n",
    "  message: newMessage,\n",
    "  chatHistory: messageHistory,\n",
    "  stream: true,\n",
    "});\n",
    "console.log(response4)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c719a935",
   "metadata": {},
   "source": [
    "### Check the response!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "f61332af-1a63-44ec-8581-999345398fb3",
   "metadata": {
    "height": 64
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "I\n",
      " apologize\n",
      " for\n",
      " the\n",
      " confusion\n",
      ".\n",
      " The\n",
      " last\n",
      " thing\n",
      " I\n",
      " mentioned\n",
      " was\n",
      " X\n",
      "HP\n",
      ",\n",
      " which\n",
      " stands\n",
      " for\n",
      " XML\n",
      " HTML\n",
      " Parser\n",
      ".\n",
      " It\n",
      " is\n",
      " an\n",
      " extension\n",
      " syntax\n",
      " for\n",
      " PHP\n",
      " that\n",
      " allows\n",
      " developers\n",
      " to\n",
      " write\n",
      " HTML\n",
      " and\n",
      " XML\n",
      "-like\n",
      " structures\n",
      " directly\n",
      " in\n",
      " their\n",
      " PHP\n",
      " code\n",
      ".\n",
      " X\n",
      "HP\n",
      " was\n",
      " created\n",
      " by\n",
      " Facebook\n",
      " and\n",
      " provides\n",
      " a\n",
      " way\n",
      " to\n",
      " write\n",
      " structured\n",
      " and\n",
      " type\n",
      "-safe\n",
      " code\n",
      " for\n",
      " building\n",
      " web\n",
      " applications\n",
      " in\n",
      " PHP\n",
      ".\n",
      " It\n",
      " served\n",
      " as\n",
      " an\n",
      " inspiration\n",
      " for\n",
      " JSX\n",
      " in\n",
      " the\n",
      " React\n",
      " ecosystem\n",
      ".\n",
      "\n"
     ]
    }
   ],
   "source": [
    "for await (const data of response4) {\n",
    "    console.log(data.response); // Print the data\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1f696259",
   "metadata": {},
   "source": [
    "\n",
    "\n",
    "## Create Llama App"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b732d778",
   "metadata": {},
   "source": [
    "### Let's see all three techniques in a real web app!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fa48c2f5",
   "metadata": {},
   "source": [
    "We will use the utility functions in **utils.ts** file. \n",
    "\n",
    "**Note:** Go to `File` and click on `Open` to access the **utils.ts** file and all the code used in this lesson."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "3f4395d9-875d-42e5-9349-ff298a6fb2e5",
   "metadata": {
    "height": 30
   },
   "outputs": [],
   "source": [
    "import { runCreateLlamaApp } from './utils.ts'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6396d481",
   "metadata": {},
   "source": [
    "**Note**: The below code might take a minute to complete. Once done, please click on the link to access the web app!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "f2c044fa-68b3-4507-bfe1-a6f11aa22b61",
   "metadata": {
    "height": 30
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "Check <a href=https://s172-31-15-23p10793.lab-aws-production.deeplearning.ai/ target='_blank'>https://s172-31-15-23p10793.lab-aws-production.deeplearning.ai/</a> you'll see a web app set up with our data! "
      ],
      "text/plan": [
       "https://s172-31-15-23p10793.lab-aws-production.deeplearning.ai/"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "runCreateLlamaApp()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "153b1eed",
   "metadata": {},
   "source": [
    "### Demonstrate streaming to the UI"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3f613678",
   "metadata": {},
   "source": [
    "Please note that the link generated above is specific to your notebook and will display the web app set up with our data! \n",
    "\n",
    "Ask it some questions about React. You'll see it instantly answers with a stream."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Deno",
   "language": "typescript",
   "name": "deno"
  },
  "language_info": {
   "file_extension": ".ts",
   "mimetype": "text/x.typescript",
   "name": "typescript",
   "nb_converter": "script",
   "pygments_lexer": "typescript",
   "version": "5.3.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}