11-machine-learning.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import random \n",
    "import math\n",
    "import scipy\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import library_data_science as lds"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction to Multi-Dimentional Data\n",
    "\n",
    "\n",
    "Multi-dimensional data can be represented as an array of tuples, where each tuple consists one, two or more elements. In this structure, the first element in each tuple is treated as independent, while the another elements typically depends on the first, reflecting the relationship between the variables. \\\n",
    "Remember that `D.size = n` and $\\forall{m < n}($ `D[m].size = k`$)$.\n",
    "\n",
    "$$D =  \\bigg< (d_{00}, d_{01}, ..., d_{0(k-1)}), (d_{10}, d_{11}, ..., d_{1(k-1)}), (d_{20}, d_{21}, ..., d_{2(k-1)}), ... , (d_{(n-1)0}, d_{(n-1)1}, ..., d_{(n-1)(k-1)}) \\bigg>,$$\n",
    "$$D = \\bigg< (D[0][0], D[0][1], ..., D[0][k-1]), (D[1][0], D[1][1], ..., D[1][k-1]), ..., (D[n-1][0], D[n-1][1], ..., D[n-1][k-1]) \\bigg>$$\n",
    "\n",
    "Multi-dimentional dataset can be unzipped to the $k$ separated sets.\n",
    "\n",
    "$$D_0 = \\big< d_{00}, d_{10}, ... , d_{(n-1)0} \\big> = \\big< D[0][0], D[1][0], ... , D[n-1][0] \\big> $$\n",
    "$$D_1 = \\big< d_{01}, d_{11}, ... , d_{(n-1)1} \\big> = \\big< D[0][1], D[1][1], ... , D[n-1][1] \\big>$$\n",
    "$$...$$\n",
    "$$D_{k-1} = \\big< d_{0(k-1)}, d_{1(k-1)}, ... , d_{(n-1)(k-1)} \\big> = \\big< D[0][k-1], D[1][k-1], ... , D[n-1][k-1] \\big>$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To demonstrate the difference, I will use the example I used in the presentation to explain what multidimensional data are.\n",
    "\n",
    "#### Example 1: Runners' Performance\n",
    "\n",
    "When examining the distances covered by runners during a 5-minute run, along with their heart rates and oxygen consumption, our data will no longer be one-dimensional. Instead, it will consist of multiple attributes for each runner.\n",
    "\n",
    "$$\n",
    "Distance\\_Heart\\_Oxygen = \\big< (1078, 145, 3.2), (896, 152, 2.9), (1196, 138, 3.5), (1009, 149, 3.1), (1078, 143, 3.3), (1096, 141, 3.4), (923, 155, 3.0) \\big>\n",
    "$$\n",
    "\n",
    "Here, each tuple represents **(distance in meters, heart rate in bpm, oxygen consumption in L/min)**, making it a **three-dimensional dataset**.\n",
    "\n",
    "$$\n",
    "Distance = \\big<1078, 896, 1196, 1009, 1078, 1096, 923\\big>\n",
    "$$\n",
    "\n",
    "$$\n",
    "Heart = \\big<145, 152, 138, 149, 143, 141, 155\\big>\n",
    "$$\n",
    "\n",
    "$$\n",
    "Oxygen = \\big<3.2, 2.9, 3.5, 3.1, 3.3, 3.4, 3.0\\big>\n",
    "$$\n",
    "\n",
    "#### Example 2: Physiological and Lifestyle Factors\n",
    "\n",
    "When studying multiple physiological and lifestyle factors influencing body weight, a more complex dataset may include height, body weight, age, and daily caloric intake.\n",
    "\n",
    "$$\n",
    "Weights\\_Heights\\_Ages\\_Calories = \\big< (60, 177, 25, 2200), (76, 189, 30, 2500), (99, 197, 28, 2700), (48, 165, 22, 1800) \\big>\n",
    "$$\n",
    "\n",
    "Each entry now contains four attributes, making it a **four-dimensional dataset**.\n",
    "\n",
    "$$\n",
    "Weights = \\big< 60, 76, 99, 48 \\big>\n",
    "$$\n",
    "\n",
    "$$\n",
    "Heights = \\big< 177, 189, 197, 165 \\big> \n",
    "$$\n",
    "\n",
    "$$\n",
    "Ages = \\big< 25, 30, 28, 22 \\big>\n",
    "$$\n",
    "\n",
    "$$\n",
    "Calories = \\big< 2200, 2500, 2700, 1800 \\big>\n",
    "$$\n",
    "\n",
    "#### Key Takeaways\n",
    "\n",
    "The more attributes we include in our dataset, the higher its dimensionality. Multidimensional data allow for deeper analysis, such as finding correlations between different factors, but they also introduce challenges like increased complexity in visualization and computational processing. \n",
    "\n",
    "**Machine learning techniques, such as Principal Component Analysis (PCA), can help reduce dimensionality while preserving essential information.**\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Machine Learning\n",
    "\n",
    "**Machine Learning (ML)** is a branch of artificial intelligence (AI) that enables computers to recognize patterns and make decisions based on data—without the need for explicitly programmed rules.\n",
    "\n",
    "![Traditional Programming versus Machine Learning](https://cdn.prod.website-files.com/614c82ed388d53640613982e/63ef5f4e24edde6ef055c3b2_traditional%20programming%20vs%20machine%20learning.jpg)\n",
    "\n",
    "### How does it work?\n",
    "\n",
    "1. **Input Data** – The ML model receives a large amount of data, such as images, text, numbers, or sounds.\n",
    "\n",
    "2. **Training** – The algorithm analyzes the data and \"learns\" relationships between them, adjusting its parameters.\n",
    "\n",
    "3. **Prediction** – After training, the model can process new data and make decisions based on it.\n",
    "\n",
    "### Examples of ML applications:\n",
    "\n",
    "- **Recommendations** (Netflix, Spotify suggesting movies/music)\n",
    "\n",
    "- **Speech and image recognition** (Siri, Google Lens)\n",
    "\n",
    "- **Spam filters** (detecting spam emails)\n",
    "\n",
    "- **Predictive systems** (weather forecasts, financial analysis)\n",
    "\n",
    "### Types of Machine Learning:\n",
    "\n",
    "1. **Supervised Learning** – The model learns from labeled data (e.g., images of cats and dogs, where it knows what is what).\n",
    "\n",
    "2. **Unsupervised Learning** – The model searches for patterns in data without labels (e.g., clustering customers based on similar behavior).\n",
    "\n",
    "![Supervised versus Unsupervised Learning](https://www.mathworks.com/discovery/machine-learning/_jcr_content/mainParsys/band/mainParsys/lockedsubnav/mainParsys/columns/a32c7d5d-8012-4de1-bc76-8bd092f97db8/image_2128876021_cop.adapt.full.medium.svg/1741205964325.svg)\n",
    "\n",
    "### Basic Paradigm of Machine Learning\n",
    "\n",
    "1. Observe set of examples: **training data**.\n",
    "\n",
    "2. Infer something about process that generated that data.\n",
    "\n",
    "3. Use inference to make predictions about previously unseen data: **test data**."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Supervised versus Unsupervised Learning\n",
    "\n",
    "Machine learning can be broadly categorized into two main types: **Supervised Learning** and **Unsupervised Learning**. The key difference lies in whether the data used for training includes labeled outputs.\n",
    "\n",
    "![Classification versus Clustering versus Regression](https://lh6.googleusercontent.com/proxy/b9cTY0TniOxMDzL0UEDPN9WdCMqxJ0ETnubKDQ37IIubX6NK1l_iGMkRZTzAdC-Xi3G2V9_jX9PlAQzsUd2g-LLxU7q0qM_KgzKiOeuIodms5uNEVQoy0xEw93U75fZPVT-R_-XN7D4h5L6E)\n",
    "\n",
    "\n",
    "## Supervised Learning\n",
    "\n",
    "Supervised learning is a type of machine learning where the model learns from **labeled data**. Each training example consists of an input and a corresponding correct output.\n",
    "\n",
    "### How it works:\n",
    "\n",
    "- The algorithm is trained on a dataset containing **inputs (X)** and **expected outputs (Y)**.\n",
    "\n",
    "- The model makes predictions and adjusts itself based on the difference between its predictions and the actual labels.\n",
    "\n",
    "- Once trained, the model can make accurate predictions on new, unseen data.\n",
    "\n",
    "### Examples:\n",
    "\n",
    "- **Email Spam Detection** – Given labeled emails (\"spam\" or \"not spam\"), the model learns to classify new emails.\n",
    "\n",
    "- **Image Classification** – Identifying whether an image contains a cat or a dog.\n",
    "\n",
    "- **Stock Price Prediction** – Predicting future stock prices based on historical labeled data.\n",
    "\n",
    "\n",
    "## Unsupervised Learning\n",
    "\n",
    "Unsupervised learning deals with **unlabeled data**. The model finds patterns and structures in the data without predefined labels.\n",
    "\n",
    "### How it works:\n",
    "\n",
    "- The algorithm analyzes input data **without any associated outputs**.\n",
    "\n",
    "- It groups similar data points or identifies hidden structures.\n",
    "\n",
    "- Often used for **clustering**, **anomaly detection**, and **pattern recognition**.\n",
    "\n",
    "### Examples:\n",
    "\n",
    "- **Customer Segmentation** – Grouping customers by purchasing behavior.\n",
    "\n",
    "- **Anomaly Detection** – Identifying fraudulent transactions in banking.\n",
    "\n",
    "- **Topic Modeling** – Discovering topics in a collection of documents.\n",
    "\n",
    "\n",
    "### Key Differences: \n",
    "| Feature              | Supervised Learning | Unsupervised Learning |\n",
    "|----------------------|--------------------|----------------------|\n",
    "| **Data Type**        | Labeled data (X, Y) | Unlabeled data (X) |\n",
    "| **Main Goal**        | Learn a mapping from inputs to outputs | Find hidden structures and patterns |\n",
    "| **Typical Use Cases** | Classification, Regression | Clustering, Anomaly Detection |\n",
    "\n",
    "![Classification versus Clustering](https://cdn.prod.website-files.com/614c82ed388d53640613982e/63ef769f6a877d715fa75008_supervised%20vs%20Unsupervised%20learning.jpg)\n",
    "\n",
    "Both types of learning have unique applications and are used depending on the problem at hand."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Feature Engineering\n",
    "\n",
    "**Feature engineering** is the process of preparing data for machine learning so the model can learn and make predictions more effectively.  \n",
    "\n",
    "First, it is important to **understand the problem and the data**. This includes **domain knowledge**, analyzing data for **missing values and anomalies**, and identifying **variable types** such as numerical or categorical.  \n",
    "\n",
    "Next, **relevant features should be selected and data transformed**. **Unnecessary columns** can be removed, **missing values handled**, and **scaling techniques applied** to adjust data for the model.  \n",
    "\n",
    "Sometimes, **creating new features** is beneficial. This can involve **calculating date differences**, **grouping values into categories** (e.g., age groups), or **combining existing variables**, such as computing price per unit.  \n",
    "\n",
    "**Categorical data** needs to be converted into numerical form, for example, by **assigning numbers to categories** or using **one-hot encoding**.  \n",
    "\n",
    "If there are **too many features**, **redundant ones should be removed**, or **dimensionality reduced** using statistical methods like **PCA**.  \n",
    "\n",
    "Finally, **changes should be validated** by comparing the model's performance **before and after modifications**. It is also crucial to **avoid overfitting**—where the model memorizes noise instead of learning general patterns.\n",
    "\n",
    "### Overfitting\n",
    "\n",
    "Overfitting occurs when a machine learning model learns the training data too well, capturing noise and random fluctuations instead of general patterns. As a result, the model performs exceptionally well on the training data but fails to generalize to new, unseen data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " # Statistical Metrics in Machine Learning\n",
    "\n",
    "When evaluating machine learning models, we use various statistical measures to assess their performance."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* **Confusion Matrix**: A table that summarizes the performance of a classification model by showing the counts of true positives, false positives, true negatives, and false negatives for multiple classes.\n",
    "\n",
    "$$\n",
    "\\begin{array}{c|cccc}\n",
    "\\text{Observed / Predicted} & c_0 & c_1 & c_2 & \\dots & c_{k-1} \\\\\n",
    "\\hline\n",
    "c_0 & Tc_0 & Fc_1 \\text{ for } c_0 & Fc_2 \\text{ for } c_0 & \\dots & Fc_{k-1} \\text{ for } c_0 \\\\\n",
    "c_1 & Fc_0 \\text{ for } c_1 & Tc_1 & Fc_2 \\text{ for } c_1 & \\dots & Fc_{k-1} \\text{ for } c_1 \\\\\n",
    "c_2 & Fc_0 \\text{ for } c_2 & Fc_1 \\text{ for } c_2 & Tc_2 & \\dots & Fc_{k-1} \\text{ for } c_2 \\\\\n",
    "\\vdots & \\vdots & \\vdots & \\vdots & \\ddots & \\vdots \\\\\n",
    "c_{k-1} & Fc_0 \\text{ for } c_{k-1} & Fc_1 \\text{ for } c_{k-1} & Fc_2 \\text{ for } c_{k-1} & \\dots & Tc_{k-1}\n",
    "\\end{array}\n",
    "$$\n",
    "\n",
    "\n",
    "* **Accuracy**: The ratio of correctly predicted instances to the total instances, measuring overall correctness but potentially misleading for imbalanced datasets.\n",
    "\n",
    "$$\n",
    "Accuracy = \\frac{\\sum_{i=0}^{k-1}{Tc_i}}{\\sum_{i=0}^{k-1}{Tc_i} + \\sum_{i=0}^{k-1}{\\sum_{j=0}^{k-1}{\\big(Fc_{i}\\text{ for } c_{j}}}\\big)}\n",
    "$$\n",
    "\n",
    "* **Precision (Positive Predictive Value)**: The proportion of correctly predicted positive instances out of all instances predicted as positive, important when false positives are costly. \n",
    "\n",
    "$$\n",
    "Precision(c_{i}) = \\frac{Tc_{i}}{Tc_{i} + \\sum_{j=0}^{k-1}{\\big(Fc_{i} \\text{ for } c_{j}\\big)}}\n",
    "$$\n",
    "\n",
    "* **Recall (Sensitivity or True Positive Rate)**: The proportion of correctly predicted positive instances out of all actual positive instances, crucial when false negatives are costly.\n",
    "\n",
    "$$\n",
    "Recall(c_{i}) = \\frac{Tc_{i}}{Tc_{i} + \\sum_{j=0}^{k-1}{\\big(Fc_{j} \\text{ for } c_{i}\\big)}}\n",
    "$$\n",
    "\n",
    "* **F1-Score**: The harmonic mean of precision and recall, balancing both metrics and useful when dealing with class imbalance.\n",
    "\n",
    "$$\n",
    "F1\\_Score(c_{i}) = \\frac{2 \\times Precision(c_i) \\times Recall(c_i)}{Precision(c_i)+ Recall(c_i)}\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_classes(C_observed: list, C_predicted: list) -> list:\n",
    "    \"\"\"\n",
    "    Returns a list of unique classes found in both observed and predicted labels.\n",
    "    \n",
    "    Parameters:\n",
    "    C_observed (list): List of observed class labels.\n",
    "    C_predicted (list): List of predicted class labels.\n",
    "    \n",
    "    Returns:\n",
    "    list: A list of unique class labels.\n",
    "    \"\"\"\n",
    "\n",
    "    classes = []\n",
    "\n",
    "    for c in C_observed:\n",
    "        if c not in classes:\n",
    "            classes.append(c)\n",
    "\n",
    "    for c in C_predicted:\n",
    "        if c not in classes:\n",
    "            classes.append(c)\n",
    "    \n",
    "    return classes\n",
    "\n",
    "\n",
    "def print_matrix(matrix):\n",
    "    \"\"\"\n",
    "    Prints a given matrix row by row.\n",
    "    \n",
    "    Parameters:\n",
    "    matrix (list of lists): The matrix to be printed.\n",
    "    \"\"\"\n",
    "\n",
    "    for row in matrix:\n",
    "        print(row)\n",
    "\n",
    "\n",
    "def confusion_matrix(C_observed: list, C_predicted: list) -> list:\n",
    "    \"\"\"\n",
    "    Computes the confusion matrix for the given observed and predicted labels.\n",
    "    \n",
    "    Parameters:\n",
    "    C_observed (list): List of observed class labels.\n",
    "    C_predicted (list): List of predicted class labels.\n",
    "    \n",
    "    Returns:\n",
    "    tuple: A tuple containing the confusion matrix (list of lists) and a dictionary \n",
    "           mapping class labels to matrix indices.\n",
    "    \"\"\"\n",
    "\n",
    "    classes = get_classes(C_observed, C_predicted)\n",
    "    indexes = {}\n",
    "\n",
    "    for i in range(len(classes)):\n",
    "        c = classes[i]\n",
    "        indexes[c] = i\n",
    "    \n",
    "    cm = [ [0] * len(classes) for i in range(len(classes)) ]\n",
    "\n",
    "    for i in range(len(C_observed)):\n",
    "        cm[indexes[C_observed[i]]][indexes[C_predicted[i]]] = cm[indexes[C_observed[i]]][indexes[C_predicted[i]]] + 1\n",
    "\n",
    "    return (cm, indexes)\n",
    "\n",
    "\n",
    "def accuracy(C_observed: list, C_predicted: list) -> float:\n",
    "    \"\"\"\n",
    "    Calculates the accuracy of the classification.\n",
    "    \n",
    "    Parameters:\n",
    "    C_observed (list): List of observed class labels.\n",
    "    C_predicted (list): List of predicted class labels.\n",
    "    \n",
    "    Returns:\n",
    "    float: Accuracy of the classification.\n",
    "    \"\"\"\n",
    "\n",
    "    cm = confusion_matrix(C_observed, C_predicted)[0]\n",
    "\n",
    "    T = 0\n",
    "\n",
    "    for i in range(len(cm)):\n",
    "        T = T + cm[i][i]\n",
    "    \n",
    "    F = 0\n",
    "\n",
    "    for i in range(len(cm)):\n",
    "        for j in range(len(cm)):\n",
    "            if not i == j:\n",
    "                F = F + cm[i][j]\n",
    "    \n",
    "    return (T) / (T + F)\n",
    "\n",
    "\n",
    "def precision(C_observed: list, C_predicted: list, c) -> float:\n",
    "    \"\"\"\n",
    "    Computes the precision for a specific class.\n",
    "    \n",
    "    Parameters:\n",
    "    C_observed (list): List of observed class labels.\n",
    "    C_predicted (list): List of predicted class labels.\n",
    "    c: The class for which precision is calculated.\n",
    "    \n",
    "    Returns:\n",
    "    float: Precision score for the specified class.\n",
    "    \"\"\"\n",
    "\n",
    "    cm, indexes = confusion_matrix(C_observed, C_predicted)\n",
    "\n",
    "    i = indexes[c]\n",
    "\n",
    "    T = cm[i][i]\n",
    "\n",
    "    F = 0\n",
    "\n",
    "    for j in range(len(cm)):\n",
    "        if not i == j:\n",
    "            F = F + cm[j][i]\n",
    "    \n",
    "    return (T) / (T + F)\n",
    "\n",
    "\n",
    "def recall(C_observed: list, C_predicted: list, c) -> float:\n",
    "    \"\"\"\n",
    "    Computes the recall for a specific class.\n",
    "    \n",
    "    Parameters:\n",
    "    C_observed (list): List of observed class labels.\n",
    "    C_predicted (list): List of predicted class labels.\n",
    "    c: The class for which recall is calculated.\n",
    "    \n",
    "    Returns:\n",
    "    float: Recall score for the specified class.\n",
    "    \"\"\"\n",
    "    cm, indexes = confusion_matrix(C_observed, C_predicted)\n",
    "\n",
    "    i = indexes[c]\n",
    "\n",
    "    T = cm[i][i]\n",
    "\n",
    "    F = 0\n",
    "\n",
    "    for j in range(len(cm)):\n",
    "        if not i == j:\n",
    "            F = F + cm[i][j]\n",
    "    \n",
    "    return (T) / (T + F)\n",
    "\n",
    "\n",
    "def f1_score(C_observed: list, C_predicted: list, c) -> float:\n",
    "    \"\"\"\n",
    "    Computes the F1-score for a specific class.\n",
    "    \n",
    "    Parameters:\n",
    "    C_observed (list): List of observed class labels.\n",
    "    C_predicted (list): List of predicted class labels.\n",
    "    c: The class for which the F1-score is calculated.\n",
    "    \n",
    "    Returns:\n",
    "    float: F1-score for the specified class.\n",
    "    \"\"\"\n",
    "\n",
    "    prec = precision(C_observed, C_predicted, c)\n",
    "    rec = recall(C_observed, C_predicted, c)\n",
    "\n",
    "    return (2 * prec * rec) / (prec + rec)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[4, 0, 0]\n",
      "[1, 2, 1]\n",
      "[1, 1, 2]\n",
      "Accuracy = 0.667\n",
      "Precision(A) = 0.667\n",
      "Precision(B) = 0.667\n",
      "Precision(C) = 0.667\n",
      "Recall(A) = 1.0\n",
      "Recall(B) = 0.5\n",
      "Recall(C) = 0.5\n",
      "F1_Score(A) = 0.8\n",
      "F1_Score(B) = 0.571\n",
      "F1_Score(C) = 0.571\n"
     ]
    }
   ],
   "source": [
    "C_obse = ['A', 'B', 'B', 'A', 'C', 'B', 'A', 'C', 'C', 'A', 'C', 'B']\n",
    "C_pred = ['A', 'B', 'A', 'A', 'C', 'C', 'A', 'A', 'C', 'A', 'B', 'B']\n",
    "\n",
    "cm = confusion_matrix(C_obse, C_pred)\n",
    "\n",
    "print_matrix(cm[0])\n",
    "\n",
    "print('Accuracy =', np.round(accuracy(C_obse, C_pred), 3))\n",
    "print('Precision(A) =', np.round(precision(C_obse, C_pred, 'A'), 3))\n",
    "print('Precision(B) =', np.round(precision(C_obse, C_pred, 'B'), 3))\n",
    "print('Precision(C) =', np.round(precision(C_obse, C_pred, 'C'), 3))\n",
    "print('Recall(A) =', np.round(recall(C_obse, C_pred, 'A'), 3))\n",
    "print('Recall(B) =', np.round(recall(C_obse, C_pred, 'B'), 3))\n",
    "print('Recall(C) =', np.round(recall(C_obse, C_pred, 'C'), 3))\n",
    "print('F1_Score(A) =', np.round(f1_score(C_obse, C_pred, 'A'), 3))\n",
    "print('F1_Score(B) =', np.round(f1_score(C_obse, C_pred, 'B'), 3))\n",
    "print('F1_Score(C) =', np.round(f1_score(C_obse, C_pred, 'C'), 3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Binary-Class Classification Metrics\n",
    "\n",
    "Let a binary example serve as an illustration, meaning true or false. In the study, we test whether the patient is infected or not. In this case, our measurements could look as follows.\n",
    "\n",
    "$$\n",
    "Confusion\\_Matrix = \\left[\\begin{array}{cc} TP & FN \\\\ FP & TN \\end{array}\\right] \\quad Accuracy = \\frac{TP + TN}{TP + TN + FP + FN}$$  \n",
    "$$Precision(\\text{Positive}) = \\frac{TP}{TP + FP} \\quad Precision(\\text{Negative}) = \\frac{TN}{TN + FN}$$\n",
    "$$Recall(\\text{Positive}) = \\frac{TP}{TP + FN} \\quad Recall(\\text{Negative}) = \\frac{TN}{TN + FP}$$"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}