Super Fast Real-time Data Processing on Cloud-Native Architecture [I] - Yaron Haviv, iguazio

Super Fast Real-time Data Processing on Cloud-Native Architecture [I] - Yaron Haviv, iguazio

While Cloud-Native architecture allows for elastic scaling and much simpler operation, it usually adds opinionated abstractions which may hinder performance or lead to implementation challenges.

This presentation will go over iguazio’s use-case for building an extremely high-performance, real-time data processing platform on the foundations of a cloud-native architecture and its 12 factors.

I will explain how we overcome networking and storage challenges of existing micro-services technologies to process data at the rate of 100Gb/s per node; address data persistency and affinity challenges; implement elastic “server-less” functions; integrate with outbound networking; and use hardware acceleration.

The session will also cover the tradeoffs and challenges we observed in working with Kubernetes, Swarm, and DC/OS, and then explain how they are addresses.

About Yaron Haviv
A serial entrepreneur with deep technological knowledge in big data, cloud, storage, networking and high-performance. Served in multiple VP and CTO roles, initiated and led several products, features and open source projects from concept to production with top to bottom and usability first approach.

[KimmoHintikka]

Pretty cool setup. How much of this can be replicated in a setup with abstract hardware stack? With abstract hardware stack, I mean setup where I can have a 100Gb pipe but cannot trust having the same hardware and therefore optimise against specific hardware.