Data Ingestion on Azure: Real-life Scenarios and Solutions by Dustin Vannoy

Session Title: Data Ingestion on Azure: Real-life Scenarios and Solutions
Speaker(s): Dustin Vannoy

Abstract: Data engineering involves more than just moving data around, but the data ingestion step is still critical to a successful data project. A foundation of the data lake or data warehouse is collecting and organizing the data. This typically involves a variety of sources each with different security or performance constraints. In this session we focus in on several common services for data ingestion and real use cases for each. We will walk through ingesting data from Azure Storage, SQL Server, APIs, and Event Hubs. We will use the best tool for each source which require using Synapse Pipelines, Synapse Serverless SQL, Synapse Spark, and Azure Databricks.