Advancing Spark - Autoloader Resource Management

Happy 2021 everybody!

To kick us off with the new year, Simon is back delving into the depths of Autoloader. Whilst it's super easy to get started with Autoloader, you can quite quickly end up with orphaned EventGrid and Blob Queue resources. The new Cloud Resource Manager Scala API lets us control these resources, and we'll take you through it in today's video.

[EvgenAnufriev]

Thank you, for detailed explanation of the Autoloader. But it's still not clear how to reload one file(for example, the last one) if I have all files in my source folder. If I drop checkpoint directory It will trigger complete reload of all files