JUC Europe 2015 - Jenkins Pipeline for Continuous Delivery of Big Data Projects by

By: Luca Milanesio, Tiago Palma and Stefano Galarraga, GerritForge

Big data is now everywhere, from mobile media analytics, banking, industry, avionics and even in medicine to monitor expansion of epidemics. In this session, Luca will show how continuous integration and continuous delivery is applied to a big data scenario that poses new challenges to the existing Jenkins framework. He will present the implementation of an agile build and deployment process used in big data software development projects for media and financial organizations in London. The talk will start with a presentation of the workflow and then will explain how existing Jenkins plugins were leveraged, as well as how integration with Docker, Mesos and the Hadoop ecosystem was achieved.