Analyst/lounes tous polluants udi dernier

analytics/notebooks/resultats_tous-polluants_udi_dernier.ipynb

@@ -0,0 +1,347 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "89d3136e",
+   "metadata": {},
+   "source": [
+    "## Derniers résultats en date - tous polluants confondus\n",
+    "\n",
+    "---\n",
+    "\n",
+    "#### 3 situations:\n",
+    "\n",
+    "VERTE: tous les paramètres sont en dessous des limites réglementaires ou recommandations.\n",
+    "\n",
+    "- Toutes les substances actives et métabolites pertinents < 0,1 µg/L et la somme des s.a + métabolites pertinents et pertinents par défaut < 0,5 µg/L et tous les métabolites non pertinents < 0,9 µg/L\n",
+    "- ET Nitrates < 50 mg/L et nitrites < 0,5 mg/L et nitrate/50 + nitrites/3 < 1 mg/L (eau conforme)\n",
+    "- ET somme des 20 PFAS < 0,1 µg/L et la sommes des 4 PFAS < 0,02 µg/L\n",
+    "- ET CVM < 0.5 µg/L\n",
+    "- ET Plomb < 0.5 µg/L\n",
+    "- ET Arsenic < 10 µg/L\n",
+    "- ET perchlorate < 4 µg/L\n",
+    "- ET 1,4-dioxane < 0.35 µg/L\n",
+    "\n",
+    "ORANGE: au moins un paramètre est au dessus de la limite réglementaire ou des recommandations.\n",
+    "\n",
+    "- Au moins 1 s.a ou métabolite pertinent ≥ 0,1 µg/L et/ou la somme des s.a + métabolites pertinents et pertinents par défaut ≥ 0,5 µg/L et/ou au moins 1 métabolite non pertinent ≥ 0,9 µg/L\n",
+    "- ET/OU Nitrates ≥ 50 mg/L et/ou nitrites ≥ 0,5 mg/L et/ou nitrate/50 + nitrites/3 ≥ 1 mg/L\n",
+    "- ET/OU somme des 4 PFAS (PFOA, PFOS, PFNA, PFHxS) ≥ 0.02 µg/L et/ou somme des 20 PFAS ≥ 0.1 µg/L\n",
+    "- ET/OU Plomb ≥ 0.5 µg/L\n",
+    "- ET/OU Arsenic ≥ 10 µg/L\n",
+    "\n",
+    "ROUGE: au moins un paramètres est au dessus d’une limite sanitaire indiquant un potentiel risque pour la santé (dans ces cas la consommation d’eau est déconseillée).\n",
+    "\n",
+    "- Au moins 1 substance active ou 1 métabolite > limite sanitaire\n",
+    "- Au moins 1 PFAS > limite sanitaire\n",
+    "- ET/OU CVM ≥ 0.5 µg/L\n",
+    "- ET/OU Plomb ≥ xx µg/L\n",
+    "- ET/OU Arsenic ≥ xx µg/L\n",
+    "- ET/OU perchlorate ≥ 4 µg/L\n",
+    "- ET/OU 1,4-dioxane ≥ 0.35 µg/L\n",
+    "\n",
+    "---\n",
+    "\n",
+    "Réalisation du modèle seulement poru CVM et PFAS pour le moment\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "b4cf0c0d",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/html": [
+       "<span style=\"None\">The &#x27;toml&#x27; package isn&#x27;t installed. To load settings from pyproject.toml or ~/.jupysql/config, install with: pip install toml</span>"
+      ],
+      "text/plain": [
+       "The 'toml' package isn't installed. To load settings from pyproject.toml or ~/.jupysql/config, install with: pip install toml"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "<span style=\"None\">Connecting to &#x27;duckdb:///../../database/data.duckdb&#x27;</span>"
+      ],
+      "text/plain": [
+       "Connecting to 'duckdb:///../../database/data.duckdb'"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "%load_ext sql\n",
+    "%sql duckdb:///../../database/data.duckdb\n",
+    "%config SqlMagic.displaylimit = 20"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "id": "2abe764d",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/html": [
+       "<span style=\"None\">Running query in &#x27;duckdb:///../../database/data.duckdb&#x27;</span>"
+      ],
+      "text/plain": [
+       "Running query in 'duckdb:///../../database/data.duckdb'"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "d1c0358c72074c44a37efd6f64139f6b",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "FloatProgress(value=0.0, layout=Layout(width='auto'), style=ProgressStyle(bar_color='black'))"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "<table>\n",
+       "    <thead>\n",
+       "        <tr>\n",
+       "            <th>categorie</th>\n",
+       "            <th>resultat</th>\n",
+       "        </tr>\n",
+       "    </thead>\n",
+       "    <tbody>\n",
+       "        <tr>\n",
+       "            <td>cvm</td>\n",
+       "            <td>inf_0_5</td>\n",
+       "        </tr>\n",
+       "        <tr>\n",
+       "            <td>cvm</td>\n",
+       "            <td>non_quantifie</td>\n",
+       "        </tr>\n",
+       "        <tr>\n",
+       "            <td>cvm</td>\n",
+       "            <td>sup_0_5</td>\n",
+       "        </tr>\n",
+       "        <tr>\n",
+       "            <td>pfas</td>\n",
+       "            <td>aucun_parametre_quantifie</td>\n",
+       "        </tr>\n",
+       "        <tr>\n",
+       "            <td>pfas</td>\n",
+       "            <td>somme_20pfas_inf_0_1_et_4pfas_inf_0_02</td>\n",
+       "        </tr>\n",
+       "        <tr>\n",
+       "            <td>pfas</td>\n",
+       "            <td>somme_20pfas_sup_0_1</td>\n",
+       "        </tr>\n",
+       "        <tr>\n",
+       "            <td>pfas</td>\n",
+       "            <td>somme_20pfas_inf_0_1_et_4pfas_sup_0_02</td>\n",
+       "        </tr>\n",
+       "        <tr>\n",
+       "            <td>pfas</td>\n",
+       "            <td>un_pfas_sup_valeur_sanitaire</td>\n",
+       "        </tr>\n",
+       "    </tbody>\n",
+       "</table>"
+      ],
+      "text/plain": [
+       "+-----------+----------------------------------------+\n",
+       "| categorie |                resultat                |\n",
+       "+-----------+----------------------------------------+\n",
+       "|    cvm    |                inf_0_5                 |\n",
+       "|    cvm    |             non_quantifie              |\n",
+       "|    cvm    |                sup_0_5                 |\n",
+       "|    pfas   |       aucun_parametre_quantifie        |\n",
+       "|    pfas   | somme_20pfas_inf_0_1_et_4pfas_inf_0_02 |\n",
+       "|    pfas   |          somme_20pfas_sup_0_1          |\n",
+       "|    pfas   | somme_20pfas_inf_0_1_et_4pfas_sup_0_02 |\n",
+       "|    pfas   |      un_pfas_sup_valeur_sanitaire      |\n",
+       "+-----------+----------------------------------------+"
+      ]
+     },
+     "execution_count": 10,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "%%sql\n",
+    "\n",
+    "SELECT DISTINCT categorie, resultat\n",
+    "FROM int__union_resultats_udi\n",
+    "WHERE periode = 'dernier_prel'\n",
+    "    AND categorie IN ('pfas', 'cvm')\n",
+    "ORDER BY categorie"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "3b316845",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/html": [
+       "<span style=\"None\">Running query in &#x27;duckdb:///../../database/data.duckdb&#x27;</span>"
+      ],
+      "text/plain": [
+       "Running query in 'duckdb:///../../database/data.duckdb'"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "0bfaa1d8fcf74f0b845c50e0044b0658",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "FloatProgress(value=0.0, layout=Layout(width='auto'), style=ProgressStyle(bar_color='black'))"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "<table>\n",
+       "    <thead>\n",
+       "        <tr>\n",
+       "            <th>cdreseau</th>\n",
+       "            <th>periode</th>\n",
+       "            <th>categorie</th>\n",
+       "            <th>resultat</th>\n",
+       "            <th>ratio</th>\n",
+       "            <th>dernier_prel_datetime</th>\n",
+       "            <th>dernier_prel_valeur</th>\n",
+       "            <th>nb_parametres</th>\n",
+       "        </tr>\n",
+       "    </thead>\n",
+       "    <tbody>\n",
+       "        <tr>\n",
+       "            <td>083001148</td>\n",
+       "            <td>dernier_prel</td>\n",
+       "            <td>pfas</td>\n",
+       "            <td>somme_20pfas_inf_0_1_et_4pfas_inf_0_02</td>\n",
+       "            <td>None</td>\n",
+       "            <td>2025-02-28 09:19:00</td>\n",
+       "            <td>None</td>\n",
+       "            <td>21</td>\n",
+       "        </tr>\n",
+       "        <tr>\n",
+       "            <td>083001148</td>\n",
+       "            <td>dernier_prel</td>\n",
+       "            <td>cvm</td>\n",
+       "            <td>non_quantifie</td>\n",
+       "            <td>None</td>\n",
+       "            <td>2025-01-31 08:50:00</td>\n",
+       "            <td>0.000</td>\n",
+       "            <td>1</td>\n",
+       "        </tr>\n",
+       "    </tbody>\n",
+       "</table>"
+      ],
+      "text/plain": [
+       "+-----------+--------------+-----------+----------------------------------------+-------+-----------------------+---------------------+---------------+\n",
+       "|  cdreseau |   periode    | categorie |                resultat                | ratio | dernier_prel_datetime | dernier_prel_valeur | nb_parametres |\n",
+       "+-----------+--------------+-----------+----------------------------------------+-------+-----------------------+---------------------+---------------+\n",
+       "| 083001148 | dernier_prel |    pfas   | somme_20pfas_inf_0_1_et_4pfas_inf_0_02 |  None |  2025-02-28 09:19:00  |         None        |       21      |\n",
+       "| 083001148 | dernier_prel |    cvm    |             non_quantifie              |  None |  2025-01-31 08:50:00  |        0.000        |       1       |\n",
+       "+-----------+--------------+-----------+----------------------------------------+-------+-----------------------+---------------------+---------------+"
+      ]
+     },
+     "execution_count": 15,
+     "metadata": {},
+     "output_type": "execute_result"
+    },
+    {
+     "ename": "",
+     "evalue": "",
+     "output_type": "error",
+     "traceback": [
+      "\u001b[1;31mThe Kernel crashed while executing code in the current cell or a previous cell. \n",
+      "\u001b[1;31mPlease review the code in the cell(s) to identify a possible cause of the failure. \n",
+      "\u001b[1;31mClick <a href='https://aka.ms/vscodeJupyterKernelCrash'>here</a> for more info. \n",
+      "\u001b[1;31mView Jupyter <a href='command:jupyter.viewOutput'>log</a> for further details."
+     ]
+    }
+   ],
+   "source": [
+    "%%sql\n",
+    "\n",
+    "SELECT *\n",
+    "FROM int__union_resultats_udi\n",
+    "WHERE periode = 'dernier_prel'\n",
+    "    AND categorie IN ('pfas', 'cvm')\n",
+    "    AND cdreseau = '083001148'\n",
+    "LIMIT 10"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "67977db8",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%%sql\n",
+    "\n",
+    "SELECT DISTINCT\n",
+    "    cdreseau,\n",
+    "    periode,\n",
+    "    (CASE\n",
+    "        WHEN categorie = 'cvm' AND resultat = 'inf_0_5'\n",
+    "            OR \n",
+    "\n",
+    "\n",
+    "\n",
+    "FROM int__union_resultats_udi\n",
+    "WHERE\n",
+    "    periode = 'dernier_prel'\n",
+    "    AND categorie IN ('cvm', 'pfas')"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": ".venv",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.12.2"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

dbt_/models/intermediate/tous_polluants/_int__tous_polluants_models.yml

@@ -0,0 +1,35 @@
+version: 1
+
+models:
+  - name: int__resultats_tous_polluants_udi_dernier
+    description: "Résultats des derniers prélèvements pour tous les polluants par UDI. Ce modèle sélectionne le dernier prélèvement pour chaque paramètre et agrège les résultats pour identifier les dépassements des seuils sanitaires et/ou règlementaires."
+    columns:
+      - name: cdreseau
+        description: "Code de l'installation (UDI)."
+        tests:
+          - not_null
+          - unique
+
+      - name: dernier_prel_datetime
+        description: "Date et heure du dernier prélèvement pris en compte dans l'analyse."
+        type: TIMESTAMP
+        tests:
+          - not_null
+
+      - name: resultat_all
+        description: "Interprétation du résultat en fonction des seuils sanitaires et règlementaires."
+        type: VARCHAR
+        tests:
+          - accepted_values:
+              values:
+                - "inf_limites"
+                - "min_1_parametre_sup_limite_reg"
+                - "min_1_parametre_sup_limite_sanitaire"
+                - "erreur"
+
+      - name: periode
+        description: "Période d'analyse des prélèvements (ici, toujours 'dernier_prel')."
+        type: VARCHAR
+        tests:
+          - accepted_values:
+              values: ["dernier_prel"]