Synchronize dependencies by using requirements.txt across the board

Diverging sets of dependencies are a constant struggle for people vs.
machines, either aiming to use the notebooks, or to validate them,
respectively.

The idea is to improve the situation by referring to the canonical
`requirements.txt` file from everywhere. In this spirit, dependency
updates can be driven well by Dependabot, and relevant software
artefacts should be synchronized better.

By using absolute URLs to the `requirements.txt` files, the relevant
`pip install` invocation should also work well on runtime environments
like Google Colab, which need them.

Author: amotl

topic/machine-learning/automl/README.md

@@ -91,7 +91,7 @@ Alternatively, you can validate all resources in this folder by invoking a
 test runner program on the top-level folder of this repository. This is the
 same code path the CI jobs are taking.
 ```shell
-pip install -r requirements.txt
+pip install -U -r requirements.txt
 ngr test topic/machine-learning/automl
 ```
 

topic/machine-learning/automl/automl_classification_with_pycaret.ipynb

@@ -115,7 +115,10 @@
    "source": [
     "## Getting started\n",
     "\n",
-    "First, install the required dependencies. "
+    "First, install the required dependencies by uncommenting and invoking the\n",
+    "`pip install` command below. Please make sure to restart the notebook runtime\n",
+    "environment afterwards. If you observe any installation problems, please report\n",
+    "them back to us."
    ]
   },
   {
@@ -124,12 +127,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "#!pip install -r requirements.txt\n",
-    "\n",
-    "# In an environment like Google Colab, please use the absolute URL to the requirements.txt file.\n",
-    "# Note: Some inconsistencies of dependencies might get reported. They can usually be ignored.\n",
-    "# Restart the runtime, if asked by Colab.\n",
-    "#!pip install -r https://raw.githubusercontent.com/crate/cratedb-examples/main/topic/machine-learning/automl/requirements.txt"
+    "#!pip install -U -r https://github.com/crate/cratedb-examples/raw/main/topic/machine-learning/automl/requirements.txt"
    ]
   },
   {

topic/machine-learning/automl/automl_timeseries_forecasting_with_pycaret.ipynb

@@ -107,7 +107,10 @@
    "source": [
     "## Getting started\n",
     "\n",
-    "First, install the required dependencies. "
+    "First, install the required dependencies by uncommenting and invoking the\n",
+    "`pip install` command below. Please make sure to restart the notebook runtime\n",
+    "environment afterwards. If you observe any installation problems, please report\n",
+    "them back to us."
    ]
   },
   {
@@ -116,12 +119,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "#!pip install -r requirements.txt\n",
-    "\n",
-    "# In an environment like Google Colab, please use the absolute URL to the requirements.txt file.\n",
-    "# Note: Some inconsistencies of dependencies might get reported. They can usually be ignored.\n",
-    "# Restart the runtime, if asked by Colab.\n",
-    "#!pip install -r https://raw.githubusercontent.com/crate/cratedb-examples/main/topic/machine-learning/automl/requirements.txt"
+    "#!pip install -U -r https://github.com/crate/cratedb-examples/raw/main/topic/machine-learning/automl/requirements.txt"
    ]
   },
   {

topic/machine-learning/llm-langchain/README.md

@@ -97,7 +97,7 @@ from scratch anytime.
 ```shell
 python3 -m venv .venv
 source .venv/bin/activate
-pip install -r requirements.txt
+pip install -U -r requirements.txt
 ```
 
 

topic/machine-learning/llm-langchain/conversational_memory.ipynb

@@ -18,7 +18,10 @@
    "source": [
     "## Prerequisites\n",
     "\n",
-    "Install required packages."
+    "First, install the required dependencies by uncommenting and invoking the\n",
+    "`pip install` command below. Please make sure to restart the notebook runtime\n",
+    "environment afterwards. If you observe any installation problems, please report\n",
+    "them back to us."
    ],
    "metadata": {
     "collapsed": false
@@ -29,7 +32,7 @@
    "execution_count": 1,
    "outputs": [],
    "source": [
-    "#!pip install -r requirements.txt"
+    "#!pip install -U -r https://github.com/crate/cratedb-examples/raw/main/topic/machine-learning/llm-langchain/requirements.txt"
    ],
    "metadata": {
     "collapsed": false

topic/machine-learning/llm-langchain/conversational_memory.py

@@ -4,7 +4,7 @@
 Synopsis::
 
     # Install prerequisites.
-    pip install -r requirements.txt
+    pip install -U -r requirements.txt
 
     # Start database.
     docker run --rm -it --publish=4200:4200 crate/crate:nightly

topic/machine-learning/llm-langchain/cratedb-vectorstore-rag-openai-sql.ipynb

@@ -52,7 +52,10 @@
    "id": "774c3e61",
    "metadata": {},
    "source": [
-    "Install required Python packages, and import Python modules."
+    "First, install the required dependencies by uncommenting and invoking the\n",
+    "`pip install` command below. Please make sure to restart the notebook runtime\n",
+    "environment afterwards. If you observe any installation problems, please report\n",
+    "them back to us."
    ]
   },
   {
@@ -62,10 +65,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "#!pip install -r requirements.txt\n",
-    "\n",
-    "# Note: If you are running in an environment like Google Colab, please use the absolute path of the requirements:\n",
-    "#!pip install -r https://raw.githubusercontent.com/crate/cratedb-examples/main/topic/machine-learning/llm-langchain/requirements.txt"
+    "#!pip install -U -r https://github.com/crate/cratedb-examples/raw/main/topic/machine-learning/llm-langchain/requirements.txt"
    ]
   },
   {

topic/machine-learning/llm-langchain/cratedb_rag_customer_support.ipynb

@@ -28,11 +28,18 @@
    },
    "source": [
     "## Getting Started\n",
-    "CrateDB supports storing vectors since version 5.5. You can leverage the fully managed service of CrateDB Cloud, or install CrateDB on your own, for example using Docker.\n",
+    "CrateDB supports storing vectors starting with version 5.5. You can leverage the\n",
+    "fully managed service of CrateDB Cloud, or install CrateDB on your own, for example\n",
+    "using Docker.\n",
     "\n",
-    "`docker run --publish 4200:4200 --publish 5432:5432 --pull=always crate:latest -Cdiscovery.type=single-node`\n",
+    "```shell\n",
+    "docker run --publish 4200:4200 --publish 5432:5432 --pull=always crate:latest -Cdiscovery.type=single-node\n",
+    "```\n",
     "\n",
-    "Install required Python packages, and import Python modules."
+    "First, install the required dependencies by uncommenting and invoking the\n",
+    "`pip install` command below. Please make sure to restart the notebook runtime\n",
+    "environment afterwards. If you observe any installation problems, please report\n",
+    "them back to us."
    ]
   },
   {
@@ -47,10 +54,7 @@
    },
    "outputs": [],
    "source": [
-    "#!pip install -r requirements.txt\n",
-    "\n",
-    "# Note: If you are running in an environment like Google Colab, please use the absolute path of the requirements:\n",
-    "#!pip install -r https://raw.githubusercontent.com/crate/cratedb-examples/main/topic/machine-learning/llm-langchain/requirements.txt\""
+    "#!pip install -U -r https://github.com/crate/cratedb-examples/raw/main/topic/machine-learning/llm-langchain/requirements.txt"
    ]
   },
   {

topic/machine-learning/llm-langchain/cratedb_rag_customer_support_langchain.ipynb

@@ -67,7 +67,10 @@
     "\n",
     "## Setup\n",
     "\n",
-    "Install required Python packages, and import Python modules."
+    "First, install the required dependencies by uncommenting and invoking the\n",
+    "`pip install` command below. Please make sure to restart the notebook runtime\n",
+    "environment afterwards. If you observe any installation problems, please report\n",
+    "them back to us."
    ]
   },
   {
@@ -83,10 +86,7 @@
    },
    "outputs": [],
    "source": [
-    "#!pip install -r requirements.txt\n",
-    "\n",
-    "# Note: If you are running in an environment like Google Colab, please use the absolute path of the requirements:\n",
-    "#!pip install -r https://raw.githubusercontent.com/crate/cratedb-examples/main/topic/machine-learning/llm-langchain/requirements.txt"
+    "#!pip install -U -r https://github.com/crate/cratedb-examples/raw/main/topic/machine-learning/llm-langchain/requirements.txt"
    ]
   },
   {

topic/machine-learning/llm-langchain/cratedb_rag_customer_support_vertexai.ipynb

@@ -67,7 +67,10 @@
     "\n",
     "## Setup\n",
     "\n",
-    "Install required Python packages, and import Python modules."
+    "First, install the required dependencies by uncommenting and invoking the\n",
+    "`pip install` command below. Please make sure to restart the notebook runtime\n",
+    "environment afterwards. If you observe any installation problems, please report\n",
+    "them back to us."
    ]
   },
   {
@@ -82,10 +85,7 @@
    },
    "outputs": [],
    "source": [
-    "#!pip install -r requirements.txt\n",
-    "\n",
-    "# Note: If you are running in an environment like Google Colab, please use the absolute path of the requirements:\n",
-    "#!pip install -r https://github.com/crate/cratedb-examples/raw/main/topic/machine-learning/llm-langchain/requirements.txt\""
+    "#!pip install -U -r https://github.com/crate/cratedb-examples/raw/main/topic/machine-learning/llm-langchain/requirements.txt"
    ]
   },
   {

topic/machine-learning/llm-langchain/document_loader.ipynb

@@ -20,7 +20,10 @@
    "source": [
     "## Prerequisites\n",
     "\n",
-    "Install required packages."
+    "First, install the required dependencies by uncommenting and invoking the\n",
+    "`pip install` command below. Please make sure to restart the notebook runtime\n",
+    "environment afterwards. If you observe any installation problems, please report\n",
+    "them back to us."
    ],
    "metadata": {
     "collapsed": false
@@ -34,7 +37,7 @@
    },
    "outputs": [],
    "source": [
-    "#!pip install -r requirements.txt"
+    "#!pip install -U -r https://github.com/crate/cratedb-examples/raw/main/topic/machine-learning/llm-langchain/requirements.txt"
    ]
   },
   {

topic/machine-learning/llm-langchain/document_loader.py

@@ -10,7 +10,7 @@
 Synopsis::
 
     # Install prerequisites.
-    pip install -r requirements.txt
+    pip install -U -r requirements.txt
 
     # Start database.
     docker run --rm -it --publish=4200:4200 crate/crate:nightly

topic/machine-learning/llm-langchain/vector_search.ipynb

@@ -38,7 +38,10 @@
    "source": [
     "## Getting Started\n",
     "\n",
-    "Install required Python packages."
+    "First, install the required dependencies by uncommenting and invoking the\n",
+    "`pip install` command below. Please make sure to restart the notebook runtime\n",
+    "environment afterwards. If you observe any installation problems, please report\n",
+    "them back to us."
    ]
   },
   {
@@ -52,7 +55,7 @@
    },
    "outputs": [],
    "source": [
-    "#!pip install -r requirements.txt"
+    "#!pip install -U -r https://github.com/crate/cratedb-examples/raw/main/topic/machine-learning/llm-langchain/requirements.txt"
    ]
   },
   {

topic/machine-learning/llm-langchain/vector_search.py

@@ -6,7 +6,7 @@
 Synopsis::
 
     # Install prerequisites.
-    pip install -r requirements.txt
+    pip install -U -r requirements.txt
 
     # Start database.
     docker run --rm -it --publish=4200:4200 crate/crate:nightly

topic/machine-learning/mlops-mlflow/README.md

@@ -48,7 +48,7 @@ way, it is easy to wipe your virtualenv and start from scratch anytime.
 ```shell
 python3 -m venv .venv
 source .venv/bin/activate
-pip install -r requirements.txt
+pip install -U -r requirements.txt
 ```
 
 

topic/timeseries/README.md

@@ -62,7 +62,7 @@ folder, also satisfying all the dependencies.
 ```console
 python3 -m venv .venv
 source .venv/bin/activate
-pip install -r requirements.txt -r requirements-dev.txt
+pip install -U -r requirements.txt -r requirements-dev.txt
 ```
 
 Then, invoke the software tests, roughly validating all notebooks within

topic/timeseries/dask-weather-data-import.ipynb

@@ -54,7 +54,12 @@
    "id": "b59cf879",
    "metadata": {},
    "source": [
-    "## Step 1: Install dependencies"
+    "## Step 1: Install dependencies\n",
+    "\n",
+    "First, install the required dependencies by uncommenting and invoking the\n",
+    "`pip install` command below. Please make sure to restart the notebook runtime\n",
+    "environment afterwards. If you observe any installation problems, please report\n",
+    "them back to us."
    ]
   },
   {
@@ -66,7 +71,7 @@
    },
    "outputs": [],
    "source": [
-    "!pip install --upgrade 'cratedb-toolkit' 'dask' 'kaggle' 'pandas==2.0.*' 'pueblo>=0.0.7' 'sqlalchemy-cratedb'"
+    "#!pip install -U -r https://github.com/crate/cratedb-examples/raw/main/topic/timeseries/requirements.txt"
    ]
   },
   {

topic/timeseries/exploratory_data_analysis.ipynb

@@ -32,7 +32,10 @@
    "source": [
     "## Getting started\n",
     "\n",
-    "To get started with this notebook install necessary dependecies and import required modules:"
+    "First, install the required dependencies by uncommenting and invoking the\n",
+    "`pip install` command below. Please make sure to restart the notebook runtime\n",
+    "environment afterwards. If you observe any installation problems, please report\n",
+    "them back to us."
    ]
   },
   {
@@ -42,10 +45,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "#!pip install -r requirements.txt\n",
-    "\n",
-    "# Note: If you are running in an environment like Google Colab, please use the absolute path of the requirements:\n",
-    "#!pip install -r https://github.com/crate/cratedb-examples/raw/main/topic/timeseries/requirements.txt"
+    "#!pip install -U -r https://github.com/crate/cratedb-examples/raw/main/topic/timeseries/requirements.txt"
    ]
   },
   {
@@ -59,10 +59,8 @@
     "warnings.filterwarnings('ignore')\n",
     "import pandas as pd\n",
     "import sqlalchemy as sa\n",
-    "import requests\n",
     "import os\n",
-    "from pycaret.time_series import TSForecastingExperiment\n",
-    "import refinitiv.data as rd"
+    "from pycaret.time_series import TSForecastingExperiment"
    ]
   },
   {

topic/timeseries/time-series-decomposition.ipynb

@@ -36,7 +36,10 @@
    "source": [
     "## Getting started\n",
     "\n",
-    "To get started with this notebook install necessary dependecies and import required modules:"
+    "First, install the required dependencies by uncommenting and invoking the\n",
+    "`pip install` command below. Please make sure to restart the notebook runtime\n",
+    "environment afterwards. If you observe any installation problems, please report\n",
+    "them back to us."
    ]
   },
   {
@@ -46,10 +49,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "#!pip install -r requirements.txt\n",
-    "\n",
-    "# Note: If you are running in an environment like Google Colab, please use the absolute path of the requirements:\n",
-    "#!pip install -r https://github.com/crate/cratedb-examples/raw/main/topic/timeseries/requirements.txt"
+    "#!pip install -U -r https://github.com/crate/cratedb-examples/raw/main/topic/timeseries/requirements.txt"
    ]
   },
   {

topic/timeseries/timeseries-anomaly-detection.ipynb

@@ -31,10 +31,10 @@
     "\n",
     "## Getting started\n",
     "\n",
-    "First, you should install the required dependencies, you can do that by uncommenting and running the next cell. Next, import the ones we will use in this exercise.\n",
-    "\n",
-    "[SQLAlchemy]: https://cratedb.com/docs/python/en/latest/sqlalchemy.html\n",
-    "[Pycaret]: https://github.com/pycaret/pycaret"
+    "First, install the required dependencies by uncommenting and invoking the\n",
+    "`pip install` command below. Please make sure to restart the notebook runtime\n",
+    "environment afterwards. If you observe any installation problems, please report\n",
+    "them back to us."
    ]
   },
   {
@@ -43,10 +43,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "%pip install -r requirements.txt\n",
-    "\n",
-    "# Note: If you are running in an environment like Google Colab, please use the absolute path of the requirements:\n",
-    "#%pip install -r https://github.com/crate/cratedb-examples/raw/main/topic/timeseries/requirements.txt"
+    "#%pip install -U -r https://github.com/crate/cratedb-examples/raw/main/topic/timeseries/requirements.txt"
    ]
   },
   {