Project Overview
This project showcases an end-to-end data engineering pipeline built on Microsoft Fabric after earning my DP-600 Certification. It demonstrates the full journey of data — from ingestion through to business intelligence reporting and automated alerting.
Architecture
The pipeline covers every layer of the modern data stack within the Microsoft Fabric ecosystem:
- Ingestion: Real-time data pulled into the Fabric environment via event streams
- Storage: Data landed and persisted in OneLake, Microsoft Fabric's unified data lake
- Transformation: PySpark notebooks in Azure Synapse Analytics used to clean, join, and model data
- Reporting: Power BI dashboards connected directly to the Fabric semantic layer for live insights
- Alerting: Data Activator configured to send automated Microsoft Teams alerts when key thresholds are breached
Key Skills Demonstrated
This project demonstrates expertise in pipeline orchestration, data modeling, real-time streaming, and BI integration — all within a single unified platform. It reflects the analytics engineering discipline of connecting raw ingestion to business-ready outputs.
Outcome
A fully automated, end-to-end data platform that requires minimal manual intervention and delivers live, trustworthy insights to decision-makers — with proactive alerts when metrics deviate from expected ranges.