Advancing Spark - Engineering behind Featurestore

In a recent video we introduced Databricks Featurestore, and ran through an example notebook to get you started with the Titanic Dataset. But... what's it actually doing when we create tables? Where is the data held? How can we get our grubby engineering fingers under the hood?

In this video, Simon takes the same example notebook and looks at applying some engineering best practices, as well as looking at the delta table that sits underneath the featurestore. That way, we can understand the impact of these commands and properly get to grips with using Featurestore in a production environment.

As always, don't forget to like & subscribe - and if you need help harnessing modern machine learning practices for your team, get in touch with Advancing Analytics.

[wshanshan]

Awesome. Always loving under the hook video.

[hubert_dudek]

Good to see you again. Healthy & prosper again :-)

[Asiagosik]

Now how can we use score_batch() on new data if we need the feature + data join to be one-hot-encoded like in this example?

[hubert_dudek]

There is no programmatic way to delete feature store table other way than through UI. If you delete delta table than you still have feature store table in inconsistent state.