High Performance Computing in a world of Data Science (& Panel discussion at the end)

Presented by Martin Callaghan, University of Leeds, UK

Universities and other research organisations have developed and used High Performance Computing (HPC) systems for a number of years to support problems solving across many computational domains including Computational Fluid Dynamics and Molecular Dynamics. Their design features such as a batch processing system and a fast interconnect make them ideal to support these often highly parallel tools and applications.

In recent years though, with the increased interest in Data Science across a number of research fields, HPC has found itself in the position of having to support quite different tools and methodologies.

In this talk, I’ll discuss the design journey we have taken for our institutional HPC, some of the Open Source projects, tools and techniques we use with our research colleagues to support Data Science problems and some of our plans for the future.

Martin Callaghan Research Computing Manager and lead the Research Software Engineering team at the University of Leeds in the UK, where we provide High Performance Computing (HPC), Programming and Software Development consultancy across a diverse research community. My role involves Research Software Engineering, training, consultancy and outreach.

He also manages a comprehensive HPC and Research Computing training programme designed to be a ‘zero to hero’ structured introduction to HPC, Cloud and research software development.

Before joining the University of Leeds, he worked as an Engineer designing machine tool control systems, a teacher and ran my own training and consultancy business. Personal research interests are in text analytics, particularly using neural networks to summarise text at scale.