Session Title: Cloud-Based ETL With Azure Data Factory And Azure Functions
Speaker: Benjamin Kettner
Abstract: Today, many ETL workloads and Data Warehouses move to the cloud. But also the data sources and requirements have changed. How can you process streaming data for a cloud-Bourne DWH? How can you pull data from REST APIs in batches? What tools do you have for processing data in the cloud? In my session I will discuss scenarios that can be solved using the Azure Data Factory, scenarios that are beyond ADF capabilities and how Azure Functions can be employed in both scenarios. I will show real-life examples utilizing Storage queues for asynchronous processing, durable entities for streaming data and orchestrator functions.
#ETL, #Azure, #ADF, #AzureFunctions, #ModernDWH, #DWH, #Cloud
300+ sessions are now available on-demand from Data Platform Summit 2021 & 2020 at no cost. Browse all sessions.
Stay tuned, more learning coming your way.