Running Jupyter Notebooks automagically in the cloud.

Running Jupyter Notebooks automagically in the cloud.

Speaker: Dr Benjamin Kettner

Jupyter notebooks have by now become the de-facto standard for many data analysts. Notebooks provide the great benefit of containing both code and documentation as well as graphics and results all in one file. However, they pose a challenge when code stemming from data analytics is supposed to be used in a productive environment. Many approaches require a rewrite of the code and thus an investment of time from either the data scientist or a developer.

In an attempt to avoid this extra work, I have developed together with a customer a way of running Jupyter Notebooks from a Git repository programmatically enabling a scenario where the results delivered by the data scientists can be included in processes and daily runs without much extra effort. In this presentation, I will explain my take on this problem. I will discuss the pros and cons of running Jupyter Notebooks in an automated way,



Dr. Benjamin Kettner is co-founder and CTO of ML!PA Consulting GmbH. Since 2020 he is Microsoft MVP and a Friend of Red Gate. He received his doctorate in applied mathematics at the Free University of Berlin in 2012. At the time of his doctorate he was a member of the DFG Research Center Matheon - Mathematics for Key Technologies and a member of the Computational Nano-Optics group at the Zuse Institute Berlin.

During his Ph.D. and studies, he worked on software development and programming as well as databases and data models. After his Ph.D. he worked as a consultant at a Microsoft BI Partner, where he was technical project manager and also managed the research and development department and worked as a software architect. In 2015 he founded ML!PA Consulting GmbH, a Microsoft partner with a focus on advanced analytics, the Internet of Things, and cloud solutions. There he works as a CTO and software architect and conceives, develops, and designs custom solutions. In 2017 ML!PA Consulting GmbH was one of only four Microsoft Partner of the Year finalists in the Internet of Things category. Dr. Benjamin Kettner is active in the Microsoft Data Platform community, leads the Berlin regional group of PASS Deutschland e.V., and frequently appears as the author of blog posts and scientific articles and speaker at technical events.