A comprehensive guide to building a modern data warehouse with SQL Server, including ETL processes, data modeling, and analytics.

A cloud-native data pipeline and visualization project analyzing Formula 1 racing data using Azure, Databricks, Delta Lake, Tableau, and Python for insightful EDA and interactive dashboards.

🦆 Batch data pipeline with Airflow, DuckDB, Delta Lake, Trino, MinIO, and Metabase. Full observability and data quality.

A production-ready PySpark project template with medallion architecture, Python packaging, unit tests, integration tests, CI/CD automation, Databricks Asset Bundles, and DQX data quality framework.

Arcane Insight is a data analytics project designed to harness the power of SQLMesh & DuckDB to collect, transform, and analyze data from Blizzard’s Hearthstone API. Focused on card statistics and attributes, this project reveals detailed insights into card mechanics, strengths, and trends to support BI and strategic analysis.

Databricks DLT Apparel Pipeline Project: Learn medallion architecture, streaming, and data engineering with Delta Live Tables. Includes synthetic data, step-by-step guide, and certification prep.

Building a modern data warehouse with SQL server, including ETL processes, data modeling, and analytics.

'Talk to Your Factory' demo leveraging Edge (Azure IoT Operations), Cloud (Microsoft Fabric), and a Factory Agent (Azure OpenAI), to streamline factory operations. It allows real-time, natural language communication with factory systems, helping operators quickly identify issues, boost efficiency, and minimize downtime.

Revolutionary AI ETL with Medallion Architecture: Zero-touch autonomous & HITL pipelines on Databricks

This project implements a Lakehouse Medallion Architecture using modern Data Stack tools such as Fivetran, Snowflake and dbt. The ficticious organization is an e-commerce company.

Unified Data Foundation with Microsoft Fabric with Options to Integrate with Azure Databricks and Microsoft Purview

Building a modern data warehouse with SQL Server, including ETL processes, data modeling and analytics

development scaffold for test driven pyspark structured streaming with fast local testing

This repo provides a step-by-step approach to building a modern data warehouse using PostgreSQL. It covers the ETL (Extract, Transform, Load) process, data modeling, exploratory data analysis (EDA), and advanced data analysis techniques.

End-to-end data pipeline transforming Olist e-commerce data through Azure cloud services. Implements medallion architecture (Bronze-Silver-Gold) with multi-source ingestion, Spark-based processing, and OLTP-to-OLAP optimization for analytics-ready datasets.

Building a Data Lakehouse using the Medallion architecture.

Extract data from many databases of Labor, Invalids and Social Affairs sectors and convert to appropriate structure and format, then upload to shared data warehouse and data mart. Thanks to that, people of state agencies can easily retrieve and analyze data based on the compiled data warehouse.

AI-Powered E-commerce Analytics is an end-to-end data engineering platform that processes e-commerce data using artificial intelligence. It extracts data from various sources, transforms it using local LLM-powered sentiment analysis, and generates automated KPIs for business insights.

End-to-end Azure Data Engineering project using Medallion Architecture, Databricks, Synapse, and Power BI.

University of Derby 2024-UDOL-2025-05-26-6CC552