Skip to content

Conversation

maxcorbeau
Copy link
Contributor

@maxcorbeau maxcorbeau commented Apr 16, 2025

🌆 BAN: villes anciennes -> nouvelles

🗺️ contexte: les villes en France qui changent de nom

💡 quoi: un DAG pour convertir les anciennes vers nouvelles villes

🎯 pourquoi: aligner vs. réalité + aider au clustering

🤔 comment:

  • nouveau champ: ville_ancienne pour affichier sur la carte ?
  • match CP + fuzzy ville: quelle niveau de fuzzy?

🖼️ Exemple

Cohorte de variations ortographiques:
image

Cohorte de ville ancienne -> nouvelle
image

🔴 Problème dbt

Peut-être suis-je fatigué, mais je ne comprends pas le comportement de mon dbt: quand je build un des 2 modèles, il me supprime l'autre. Pourtant dans les logs dbt je ne vois q'1 DROP sur le modèle en cours de build

# build _typo & check both views
echo -e "\n📦 DBT BUILD: typo"
dbt build --select marts_enrich_acteurs_villes_suggest_typo --debug | grep -E '(DROP|CREATE).*VIEW' -A 1
echo "🔎 CHECK PSQL: ANY suggest view"
psql -h localhost -U qfdmo -d qfdmo -p 6543 -Atc "SELECT schemaname || '.' || viewname FROM pg_views WHERE viewname LIKE '%marts_enrich_acteurs_villes_suggest%';"
# build _new & check both views
echo -e "\n📦 DBT BUILD: new"
dbt build --select marts_enrich_acteurs_villes_suggest_new --debug | grep -E '(DROP|CREATE).*VIEW' -A 1
echo "🔎 CHECK PSQL: ANY suggest view"
psql -h localhost -U qfdmo -d qfdmo -p 6543 -Atc "SELECT schemaname || '.' || viewname FROM pg_views WHERE viewname LIKE '%marts_enrich_acteurs_villes_suggest%';"

image

En fin de script je m'attendrais à avoir 2 vues:

  • public.marts_enrich_acteurs_villes_suggest_typo
  • public.marts_enrich_acteurs_villes_suggest_new

📆 Prochaines PRs

  • Ecrire les schemas DBT
  • Résoudre 🔴 Problème dbt

@maxcorbeau maxcorbeau force-pushed the enrich_ban_new_cities branch 2 times, most recently from 27f850f to cc0d305 Compare April 17, 2025 07:41
@maxcorbeau maxcorbeau changed the base branch from main to enrich_acteurs_closed_replace_v1 April 17, 2025 07:41
@maxcorbeau maxcorbeau force-pushed the enrich_ban_new_cities branch from 4522f78 to 2f1a4e3 Compare April 17, 2025 08:02
@maxcorbeau maxcorbeau marked this pull request as ready for review April 17, 2025 11:51
@maxcorbeau maxcorbeau requested a review from a team as a code owner April 17, 2025 11:51
@maxcorbeau maxcorbeau requested review from kolok and fabienheureux and removed request for a team April 17, 2025 11:51
@maxcorbeau maxcorbeau force-pushed the enrich_acteurs_closed_replace_v1 branch from e106fc0 to ba49c98 Compare April 21, 2025 07:39
@maxcorbeau maxcorbeau added enhancement New feature or request dbt labels Apr 21, 2025
@maxcorbeau maxcorbeau changed the title 🌆 BAN: villes anciennes -> nouvelles 🌆 [ENRICH] BAN: villes anciennes -> nouvelles Apr 21, 2025
@maxcorbeau maxcorbeau force-pushed the enrich_ban_new_cities branch from 21a04b0 to 40f02e4 Compare April 21, 2025 12:01
@maxcorbeau maxcorbeau force-pushed the enrich_acteurs_closed_replace_v1 branch 2 times, most recently from 09afc1d to b233cf2 Compare April 24, 2025 05:27
@maxcorbeau maxcorbeau force-pushed the enrich_ban_new_cities branch from 9e40459 to abfe6cf Compare April 24, 2025 05:39
@maxcorbeau maxcorbeau changed the base branch from enrich_acteurs_closed_replace_v1 to main April 24, 2025 05:40
@maxcorbeau maxcorbeau changed the base branch from main to enrich_acteurs_closed_replace_v1 April 24, 2025 05:40
@maxcorbeau maxcorbeau force-pushed the enrich_acteurs_closed_replace_v1 branch from dca5e56 to 947e530 Compare April 24, 2025 10:20
Base automatically changed from enrich_acteurs_closed_replace_v1 to main April 24, 2025 10:23
@maxcorbeau maxcorbeau force-pushed the enrich_ban_new_cities branch from abfe6cf to 8fa0d32 Compare April 24, 2025 12:53
Copy link
Contributor

@kolok kolok left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quelques commentaires non bloquants

@maxcorbeau maxcorbeau force-pushed the enrich_ban_new_cities branch from b13ca01 to 8df023d Compare April 28, 2025 04:52
@maxcorbeau maxcorbeau enabled auto-merge (squash) April 28, 2025 06:48
@fabienheureux fabienheureux disabled auto-merge April 28, 2025 07:55
@fabienheureux fabienheureux merged commit 9dcb8db into main Apr 28, 2025
5 checks passed
@fabienheureux fabienheureux deleted the enrich_ban_new_cities branch April 28, 2025 07:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dbt enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants