From Legacy On-Premises EDW to a Scalable Azure & Microsoft Fabric Platform
About the Client
Supply Medium is a technology-driven organization specializing in cloud, data engineering, analytics, and enterprise data solutions. As its operations and client base expanded, the company required a modern data platform capable of supporting real-time analytics, self-service reporting, and future AI-driven initiatives.
Background
Supply Medium had built a mature on-premises Enterprise Data Warehouse (EDW) using SQL Server, SSIS for ETL, and reporting through Power BI, Tableau, and Excel. While the platform reliably supported traditional reporting, it struggled to meet growing business demands for real-time insights, scalability, and advanced analytics.
As data volumes increased and business requirements evolved, the organization recognized the need to modernize its architecture with a cloud-native platform capable of supporting enterprise-scale analytics and future AI/ML workloads.
Challenges
Business Challenges
- Data was refreshed only once per day, limiting real-time decision-making.
- EDW access was restricted to a small group of analysts.
- Data quality issues from source systems impacted reporting accuracy.
- No scalable foundation for AI, machine learning, or advanced analytics.
Technical Challenges
- Slow EDW query performance and lengthy ETL processing.
- Scaling SQL Server, SSIS, and Power BI infrastructure became increasingly expensive.
- Legacy on-premises technology had limited vendor support and flexibility.
- Growing data volumes placed significant pressure on existing infrastructure.
To overcome these limitations, Supply Medium adopted a phased migration strategy using Microsoft Azure and Microsoft Fabric, implementing a Lambda Architecture to support both batch and real-time data processing.
Project Objectives
The primary objective was to replace the legacy on-premises Enterprise Data Warehouse with a modern cloud-native data platform capable of supporting enterprise analytics at scale.
Key goals included:
- Azure-hosted, highly scalable infrastructure.
- Support for both batch and real-time data processing.
- Built-in data quality validation framework.
- Secure role-based access control (RBAC).
- Support for structured, semi-structured, and unstructured data.
- Unified Power BI reporting with self-service analytics.
- Future-ready architecture for AI and machine learning initiatives.
Our Solution: Microsoft Fabric
The migration followed a phased implementation approach centered on Microsoft Fabric, delivering a unified cloud data platform built on Lambda Architecture.
Lambda Architecture
Batch Layer
- Azure Data Factory and Microsoft Fabric Notebooks managed ELT pipelines.
- Data was processed using the Medallion Architecture (Bronze → Silver → Gold).
Speed Layer
- Microsoft Fabric Event Stream enabled near real-time ingestion through Azure Event Hub.
- Data was processed into KQL Databases and Lakehouse Delta Tables.
Serving Layer
- T-SQL and Spark endpoints exposed trusted datasets to Power BI, Excel, and downstream consumers.
All enterprise data was standardized using Delta Lake within Microsoft OneLake, eliminating redundant storage across compute engines.
Additional capabilities included:
- Microsoft Purview integration for governance and lineage.
- Microsoft Fabric F128 capacity for enterprise-scale workloads.
- CI/CD pipelines for automated deployments.
- Enterprise-wide Data Quality Framework implemented from Phase 1.
Phase 1: Oracle Data Migration
The first phase focused on migrating Oracle-based Enterprise Data Warehouse workloads from on-premises infrastructure into Microsoft Fabric.
Data Pipeline
Oracle Object Storage
↓
Azure Blob Storage (Azure Data Factory)
↓
Microsoft Fabric Bronze Layer
↓
Microsoft Fabric Silver Layer
↓
Microsoft Fabric Gold Layer
The migration included:
- Oracle General Ledger (GL) datasets.
- Oracle Discover datasets.
- Common reference data.
- Historical data migration.
- End-to-end ETL pipelines.
- Power BI dashboard migration.
Success Criteria
- Cloud data fully matched on-premises EDW metrics.
- Power BI dashboards achieved complete reporting parity.
- ETL runtime improved beyond the existing eight-hour processing window.
- Legacy Oracle ETL pipelines were successfully retired.
Phase 2: Platform Expansion
Following Oracle migration, the platform expanded to include operational and workforce systems.
Additional workloads included:
- CRM platforms.
- Workforce management systems.
- Survey and API-based datasets.
The implementation delivered:
- 47 Bronze Layer tables.
- 47 Silver Layer tables.
- 30 Gold Layer tables.
Additional work included:
- Historical data migration.
- Complete pipeline development.
- Power BI migration.
- CI/CD framework expansion.
- Proof-of-concepts for SharePoint Shortcuts and orchestration automation.
Success Criteria
- Complete data and reporting parity.
- Faster ETL processing.
- Successful production deployment.
- Comprehensive documentation and user acceptance testing.
Future Roadmap
Phase 3
- Data quality improvements.
- Reliability enhancements.
- Data model optimization.
Phase 4
- Migration of remaining enterprise data sources.
- Complete reporting migration.
- Full retirement of the on-premises Enterprise Data Warehouse.
Business Value
Operational Improvements
- Faster and more reliable ETL execution.
- Horizontally scalable cloud infrastructure.
- Flexible service-level agreements based on business requirements.
- Enterprise-wide access to trusted data.
Data Governance
- Embedded Data Quality Framework.
- Microsoft Purview integration.
- Platform-level Role-Based Access Control (RBAC).
- Improved historical data consistency.
Self-Service Analytics
- Power BI certified datasets.
- Excel connectivity for business users.
- Enterprise-scale ad hoc analytics.
- Greater user adoption through self-service reporting.
Future Readiness
- AI and machine learning-ready platform.
- Support for structured, semi-structured, and unstructured data.
- Scalable architecture supporting future modernization initiatives.
Ongoing Support Services
Following migration, ongoing support included:
Platform Operations
- Load monitoring.
- ETL support.
- Performance optimization.
- Issue resolution.
Data Management
- New data source integration.
- Data model enhancements.
- Business-driven data updates.
Reporting Services
- Development of new Power BI reports.
- Enhancement of existing dashboards.
- Dataset maintenance and optimization.
The support model allows the delivery team to scale resources as project demands increase.
Results
The phased migration enabled Supply Medium to successfully modernize its Enterprise Data Warehouse using Microsoft Fabric and Azure. The organization achieved significantly improved scalability, faster ETL performance, stronger governance, enhanced self-service analytics, and a future-ready cloud platform designed to support enterprise reporting, real-time analytics, and AI-driven innovation.