Advancing Spark - Azure Databricks News July 2022

Given we had Data + AI Summit, we have rolled June and July's Databricks Newscast together into a bumper edition! There are some nice little UI improvements, the GA of Photon with some extra functionality, Databricks Runtime 11.1 which has some HUGE improvements, plus some extra goodies!

We'll be covering both platform releases, and the new Databricks Runtime 11.1, all in one lovely video.

Don't forget to Like & Subscribe if you're new round here, and get in touch if you need some help on your Advanced Analytics journey

[ArcaLuiNeo]

When you'll cover Serverless SQL, perhaps you can cover the network security and how to secure data overall...

[cweymouth]

Hey, has anyone had issues with the "New Visualizations" engine for Databricks notebooks not showing? Its almost 3 months since release and this is still not available in our Azure Databricks notebooks. I've tried different runtimes (9.1 ML, 10.4 ML, 11.0 ML). Are these scoped to standard runtimes? Do I need photon acceleration? Getting this would greatly improve some workflow.

[danhorus]

Really curious about Serverless SQL Warehouse. Hope they can somehow add support to external Hive metastores and VNETs

[jordanfox]

Isn't there an argument for landing data into raw/landing zone in a human readable format for better future troubleshooting?