Introducing U-SQL; Part 2 of 3: Scaling U-SQL and doing SQL in U-SQL

Speaker: Michael Rys

Abstract

Making Big Data processing easy requires a great developer support that hides the complexity of managing the scale, allows to easily integrate custom code to handle the complex processing requirements ranging from data cleanup to advanced processing of unstructured data, and provides great tool support for the developer to help in the iterative development process. Thus when we at Microsoft introduced the Azure Data Lake, we decided to also include a new language called U-SQL to make Big data processing easy. It unifies the declarative power of SQL and the extensibility of C# to make writing custom processing of big data easy. It also unifies processing over all data – structured, semistructured and unstructured data – and queries over both local data and remote SQL data sources. This presentation will give you an overview on U-SQL, why we decided to build a new language, what its core philosophical underpinnings are as well as show the language in its natural habitat – the development tooling – showing the language capabilities as well as the tool support from starting your first script to analyzing its performance.



Bio

Michael has been doing data processing and query languages since the 1980s. Among other things he has been representing Microsoft on the XQuery and SQL design committees and has taken SQL Server beyond relational with XML, Geospatial and Semantic Search. Currently he is working on Big Data query languages such as SCOPE and U-SQL when he is not enjoying time with his family under water, on the ski slopes, or at autocross.