Graph Features in Spark 3 0 Integrating Graph Querying and Algorithms in Spark Graph- Mats Rydberg

Spark 3.0 introduces a new module: Spark Graph. Spark Graph adds the popular query language Cypher, its accompanying Property Graph Model and Graph Algorithms to the data science toolbox. Graphs have a plethora of useful applications in recommendation, fraud detection and research. The tutorial aims to help understanding when graphs should be used and how Spark Graph can be used to extend analytical workflows. In this tutorial we will explore the concepts and motivations behind graph querying and graph algorithms, the components of the new Spark Graph module and their APIs, and how those APIs allow you to successfully write your own graph applications and integrate them in your data science workflows. The tutorial is a mixture of presentation, code examples, and notebooks. We will demonstrate how to write an end-to-end Graph application that operates on different kinds of input data. We will show how Spark Graph interacts with Spark SQL and openCypher Morpheus, a Spark Graph extension that allows you to easily manage multiple graphs and provides built-in Property Graph Data Sources for the Neo4j graph database as well as Cypher language extensions. At the end of the tutorial, attendees will have a good understanding of when to apply graphs in their data science workflows, how to bring Spark Graph into an existing Spark workflow and how to make best use of the new APIs. This tutorial will be both lead by the presenters and also hands-on interactive session. The tutorial material will be made available during the presentation.

About: Databricks provides a unified data analytics platform, powered by Apache Spark™, that accelerates innovation by unifying data science, engineering and business.

Connect with us:

[chenrianglam]

Is there any reason why this feature is not delivered in Spark 3?

[anushjawahar]

Also, is there any limitation on the size of graphs to be used for n-hop traversals? Does it also provide Pregel like traversals that comes along with Graphx?

[anushjawahar]

Why is only dataframe supported and not dataset?

[ashisparajuli]

How to visualize the data without using neo4j like databases?