Spring and Big Data

Recorded at SpringOne Platform 2016.
Speaker: Thomas Risberg

In this talk we will discuss ways to develop big data pipelines using Spring technologies. Learn how we can stream data into HDFS, run a Spark or a Hive job and extract the results from HDFS or Cassandra for presentation. The solution we develop will be a cloud-native pipeline that we will be able to run both locally and in the cloud.

The presentation focuses around the code for our solution and we also cover how to set up a test environment both locally and in the cloud. While building our solution using Spring Boot and Spring for Apache Hadoop we’ll cover how this app can be integrated in a larger context such as Spring Cloud Stream and Spring Cloud Data Flow. We will also discus configuration solutions for running our app in a variety of environments.

After attending this talk you will have an understanding of how a combination of Spring projects can help build Big Data solutions that incorporate and orchestrate many diverse technologies. You will also better understand how these solutions can be built to run in a cloud environment as well as locally on your development laptop.

[Strannik]

It's a shame not to test all the steps of your demo before the talk