You know SQL? Then You’re set for real-time data processing with Azure Stream Analytics by Florian Eiden

Session Title: You know SQL? Then You’re set for real-time data processing with Azure Stream Analytics
Speaker(s): Florian Eiden

Abstract: You know SQL? You need to process data in real time? Azure Stream Analytics (ASA) is the service for you. But let’s dig a bit deeper and understand what makes a streaming query in ASA different from a T-SQL one in SQL Server.
In this session we’ll start by a short introduction to stream processing and its typical applications like Streaming ETL or real-time dashboarding, Then we’ll focus on the 3 key aspects that we need to keep in mind when working on our streaming SQL queries in ASA.

First we’ll talk data types. We’ll see how to best handle the complex ones (JSON nested records and array), but also that ASA applies types at the row level (which is both awesome and treacherous) and how to deal with it.

Then we’ll discuss the temporal aspect of stream processing, in theory and how it’s implemented in ASA. That means time skew policies, windowing functions and how timestamps evolve as data flows through query steps.

Finally we’ll have a performance conversation. We’ll see how partitioning a stream differs (or not) from partitioning a table. We’ll take a look at the metrics to monitor and how/when to scale an ASA job.