An Azure Data Engineer's ETL Toolkit - Simon Whiteley

Many organisations have adopted a "cloud first" strategy, designing sophisticated data architectures in Azure - but there's already a wealth of existing business logic, cleansing, and transformation in your existing on-premise SSIS packages. A direct lift and shift of SSIS into the cloud is a tempting solution, but is it the best answer? This session serves as an introduction to the various data movement and transformation options available in Azure, for aspiring data engineers, data architects, and anyone working with data in the cloud. We’ll compare and contrast SSIS with other tools such as Data Factory, Databricks, and Logic Apps, and showcase real-world examples of where each tool fits in. We will then take a focus on Data Factory, looking at some common ETL patterns that can take advantage of the cloud and demonstrating how to put them to best use, including the new Data Flow features that will take your cloud ETL to the next level. By the end of this session, you will have a better understanding of the limitations of SSIS in Azure, a deeper knowledge of how Azure Data Factory can fit into your cloud migration plans, and a roadmap for moving towards truly scalable cloud ETL in an achievable way. This session is the third session in the Becoming an Azure Data Engineer Learning Pathway. For more information please visit our Learning Pathways page.