About the Client
The client is a leading provider of event technology and production services, supporting meetings and events across multiple regions. Its services span audiovisual production, digital solutions, and on-site event support for venues, organizers, and enterprise customers.
Operating at scale, the organization manages significant volumes of data across event operations, customer engagement, workforce management, finance, and service delivery. This data plays a critical role in operational reporting, business intelligence, and strategic decision-making.
Background
The client had built a mature on-premises Enterprise Data Warehouse (EDW) on Microsoft SQL Server, using SSIS for ETL and Tableau, Power BI, and Excel for reporting and analytics.
While the environment reliably supported traditional analytics, growing business requirements exposed limitations around scalability, data freshness, performance, and advanced analytics. Data was typically a day old, access to the EDW was restricted to a relatively small analyst community, and scaling the database, SSIS workloads, and Power BI gateways was becoming increasingly costly and complex.
The organization needed a modern, cloud-native platform capable of supporting both batch and near-real-time processing, expanding self-service analytics, improving governance, and creating a foundation for future AI and machine learning workloads.
To achieve this, the client initiated a phased migration from its legacy EDW to a modern data ecosystem powered by Microsoft Azure and Microsoft Fabric, with Supply Medium supporting the transformation.
The Challenge
The existing environment presented a combination of business and technical constraints that limited the organization’s ability to scale its analytics capabilities.
Business Challenges
Delayed Data Availability: Business data was typically one day old, limiting the ability to make timely, data-driven decisions.
Limited EDW Accessibility: Access was restricted to a small analyst community, preventing broader adoption of self-service analytics.
Data Quality Concerns: Source-level inconsistencies occasionally affected the reliability and quality of downstream reporting.
Limited Advanced Analytics: The legacy environment offered no practical path for supporting AI, machine learning, data mining, or experimentation at scale.
Technical Challenges
Performance Bottlenecks: Slow EDW queries and inefficient ETL workloads increased processing times and delayed reporting.
Complex and Costly Scaling: Expanding SQL Server, SSIS, and Power BI gateway infrastructure required significant operational effort and investment.
Aging Technology Stack: Parts of the on-premises environment were approaching the limits of vendor support and long-term viability.
Infrastructure Constraints: The existing architecture could not scale horizontally to meet growing workloads and evolving analytics requirements.
Transformation Objectives
The primary objective was to retire the on-premises EDW and replace it with a scalable, cloud-native data platform that could support current analytics requirements while creating a foundation for future growth.
Key objectives included:
- Establishing highly available, horizontally scalable infrastructure on Azure
- Supporting batch and near-real-time processing based on business-specific requirements
- Embedding data quality validation into the platform
- Expanding access through role-based access control (RBAC)
- Supporting structured, semi-structured, and unstructured data
- Creating a unified and certified Power BI reporting environment
- Enabling enterprise-wide self-service analytics
- Establishing platform readiness for future AI/ML workflows and experimentation
The Solution: Microsoft Fabric
Supply Medium designed a structured, phased transformation strategy centered on Microsoft Fabric, Azure services, and a Lambda Architecture capable of supporting both batch and near-real-time workloads.
The architecture was organized into three core processing layers.
Batch Layer
ELT pipelines built with Azure Data Factory and Fabric Notebooks process source data through a medallion architecture:
Bronze → Silver → Gold
The Bronze layer captures raw source data, the Silver layer applies cleansing and transformations, and the Gold layer produces curated, business-ready datasets for analytics and reporting.
Speed Layer
Microsoft Fabric Eventstream enables near-real-time ingestion through Azure Event Hubs, with data processed into KQL databases and Lakehouse Delta tables.
This layer provides a foundation for use cases requiring faster data availability than traditional batch processing.
Serving Layer
T-SQL and Spark endpoints expose curated data to Power BI, Excel, and other downstream consumers, enabling different user groups to access trusted data using familiar analytics tools.
Data across the platform is standardized using Delta Lake format within OneLake, reducing unnecessary duplication across compute engines and creating a consistent data foundation.
Microsoft Purview integration strengthens governance, discovery, and lineage visibility across the data estate.
The platform was sized at F128 Microsoft Fabric capacity, exceeding the original F64 baseline estimate, to support additional workloads and anticipated future growth.
DevOps and CI/CD pipelines, together with a standardized data quality framework, were established during Phase 1 and designed to serve as foundational standards for every subsequent migration phase.
Phase 1: Oracle Data Migration
The first phase focused on migrating the client’s Oracle EDW workloads, associated ETL processes, and Power BI reporting to the cloud.
Oracle was selected as the initial workload because it was relatively loosely coupled with the broader EDW, minimizing dependencies and providing a practical starting point for establishing the cloud migration blueprint.
The phase also required transitioning the source layer from Oracle ATP to Oracle Object Storage.
Two primary Oracle data streams were included:
GL – General Ledger
Discover
Both streams operated as separate batches in the legacy EDW and were independently managed within the new cloud architecture.
Data Pipeline Architecture
Oracle Object Storage → Azure Blob Storage via ADF → Fabric Bronze → Fabric Silver → Fabric Gold
Each medallion layer serves a specific purpose.
Bronze Layer: Ingests raw source files with minimal modification.
Silver Layer: Cleans and transforms data to align with the existing Oracle ATP table structures and business requirements.
Gold Layer: Produces final curated datasets that mirror the required structures and business logic of the legacy EDW.
The migration covered Oracle ERP data for GL and Discover, along with shared reference datasets including LocationList, WLCEmployee, and RLS.
Historical data was migrated, end-to-end pipelines were established, and associated Power BI dashboards were transitioned to the new cloud platform.
Phase 1 Success Criteria
The phase was considered successful when:
- On-premises EDW and cloud platform data matched across row counts, aggregates, and key business metrics
- Power BI dashboards powered by the cloud platform achieved parity with their on-premises equivalents
- Cloud ETL runtime matched or improved upon the existing average runtime of approximately 8 hours
Phase 1 Exit Milestone
Following successful validation, the on-premises ETL processes supporting Oracle sources could be permanently decommissioned.
Phase 2: Expansion & Platform Scaling
The second phase extended the migration to additional operational and workforce data sources, including:
Compass: CRM data
NWF: Workforce data
Medallia: Survey and API-based data
This phase expanded the batch processing layer and moved a broader range of ETL and reporting workloads from on-premises infrastructure to the cloud.
Across a 19-week timeline, the implementation covered:
47 Bronze tables
47 Silver tables
30 Gold tables
The scope included source connectivity, historical data migration, pipeline development across Bronze, Silver, and Gold layers, Power BI dashboard migration, and extension of the CI/CD framework established during Phase 1.
The team also conducted proofs of concept for SharePoint Shortcuts and custom Airflow-style orchestration to evaluate additional integration and workflow capabilities.
Phase 2 Success Criteria
Success focused on achieving data and dashboard parity while improving ETL performance and progressively transitioning workloads to the cloud.
Key workstreams advanced from initial proofs of concept through source integration, pipeline development, reporting migration, UAT, production deployment, and documentation.
Future Migration Phases
The transformation roadmap was designed as a four-phase program, with each stage building on the technical and governance foundation established in previous phases.
Phase 1 — Oracle Migration
Focus: Oracle GL and Discover migration
Objective: Move Oracle ETL workloads and associated Power BI dashboards to the cloud.
Exit Condition: Oracle on-premises ETL is decommissioned.
Phase 2 — Bronze & Silver Migration
Focus: Remaining source systems
Objective: Ingest and transform source data into cloud-based Bronze and Silver layers.
Exit Condition: Cloud Bronze and Silver layers are validated while the legacy EDW continues operating during transition.
Phase 3 — Gold Layer & Model Refinement
Focus: Dimensions, facts, summaries, data quality, and model refinement
Objective: Build the curated Gold layer for migrated sources and strengthen reliability and business-ready data models.
Exit Condition: Full cloud EDW data parity is achieved.
Phase 4 — Reporting Cutover
Focus: Enterprise reporting migration
Objective: Transition remaining Power BI dashboards and datasets to the cloud platform.
Exit Condition: The legacy on-premises EDW is fully retired.
Business Value
The Azure and Microsoft Fabric transformation creates a scalable data foundation capable of supporting the client’s evolving operational, analytical, and innovation requirements.
Operational Efficiency
Modernized pipelines enable more reliable and timely ETL completion for field and operational reporting.
Horizontal scalability removes traditional infrastructure ceilings, while differentiated processing patterns allow data sources to operate according to their specific business SLAs.
The cloud architecture also supports a significantly broader user community without requiring equivalent expansion of on-premises infrastructure.
Data Quality & Governance
An embedded Data Quality Framework improves validation and consistency throughout the data lifecycle.
Microsoft Purview integration strengthens governance, lineage, and data discovery, while platform-level RBAC enables controlled access across different user groups.
Historical data is retained consistently, providing teams with reliable information for reporting and longitudinal analysis.
Self-Service Analytics
The modern platform expands access to trusted data across the organization.
Business users can continue using Excel for everyday analysis, while Power BI provides certified enterprise reporting alongside flexible ad-hoc analytics.
The architecture also enables users to explore data and experiment without depending entirely on centralized reporting teams.
Future-Ready Data Platform
Support for structured, semi-structured, and unstructured data expands the range of workloads the organization can bring into its analytics ecosystem.
Built-in readiness for AI and machine learning creates a foundation for future predictive analytics, intelligent automation, and advanced data science initiatives.
Most importantly, the architecture established during the initial phases provides a repeatable blueprint for completing the broader EDW modernization program.
Ongoing Support Services
Following migration, Supply Medium provides ongoing support across the cloud data platform and reporting environment to maintain reliability, performance, and alignment with evolving business requirements.
Platform Maintenance
Support includes load monitoring, ETL and data issue resolution, pipeline maintenance, and ongoing performance optimization.
Data Management
As business requirements evolve, new data points can be incorporated into the platform while existing datasets and transformation logic can be updated as needed.
Reporting & Analytics
Support extends to developing new reports, enhancing existing dashboards, and updating datasets to accommodate changing reporting requirements.
The delivery team can also be augmented when project demand exceeds steady-state support capacity.