Best Practices for Building Robust Data Platform with Apache Spark and Delta

This talk will focus on Journey of technical challenges, trade offs and ground-breaking achievements for building performant and scalable pipelines from the experience working with our customers. The problems encountered are shared by many organizations and so the lessons learned and best practices are widely applicable.

These include:

- Operational tips and best practices with Apache Spark in production
How your choice of overall data pipeline design influences performance and outcomes.
- Common misconfigurations that prevent users from getting the most out of their Apache Spark deployment.
- Attendees will come out of the session with Best Practices and Strategies that can be applied to their Big Data architecture, such as:

Optimizing Cost to Drive Business Value

- Achieving Performance at Scale with Apache Spark and Delta Lake
- Ensuring security guarantees including, recommendations on handling GDPR and CCPA requests
- Audience: The attendees should have some knowledge of setting up the Big Data Pipelines and Apache Spark.

About:
Databricks provides a unified data analytics platform, powered by Apache Spark™, that accelerates innovation by unifying data science, engineering and business.

Connect with us: