Designed by Agile Lab, Witboost is a versatile platform that addresses a wide range of sophisticated data engineering challenges. It enables businesses to discover, enhance, and productize their data, fostering the creation of automated data platforms that adhere to the highest standards of data governance. Want to know more about Witboost? Check it out here or contact us!.
- Overview
- HLD
- Running
- Testing
- Building
- Local Dev Setup
- Poetry
- Settings Management
- Metamodel
- Docker
- Open Telemetry
- Authentication
The project contains a definition of a Purview Data Catalog Plugin implemented as a FastAPI Python microservice, tailored for the Witboost platform. The Data Catalog Plugin is inserted in the deployment process such that just before deploying a Data Product this microservice will be called by Witboost and the Data Product's descriptor will be uploaded into Microsoft Purview.
This Data Catalog Plugin creates the metamodel for the hierarchy Domain -> Data Product -> Component (Output Ports and Workloads). For Output Ports it also creates the physical schema and, if the information is present, link it to the related storage account.
If you have a script for executing the project, you can execute the app with the following command without the need to manually enter the virtual environment.
poetry run server_start.shOr you can execute
poetry run uvicorn src.main:app --host localhost --port 5002 --log-config logging.yamlIf you want to execute unit tests you can do it by executing:
poetry run python -m pytest tests/unitIntegration tests are also available but require authentication. See Integration testing for more information.
Build the project and generate the wheel by executing:
poetry buildWitboost is a cutting-edge Data Experience platform, that streamlines complex data projects across various platforms, enabling seamless data production and consumption. This unified approach empowers you to fully utilize your data without platform-specific hurdles, fostering smoother collaboration across teams.
It seamlessly blends business-relevant information, data governance processes, and IT delivery, ensuring technically sound data projects aligned with strategic objectives. Witboost facilitates data-driven decision-making while maintaining data security, ethics, and regulatory compliance.
Moreover, Witboost maximizes data potential through automation, freeing resources for strategic initiatives. Apply your data for growth, innovation and competitive advantage.
Contact us or follow us on: