Azure Data Factory with Databricks setup connection

Both, Azure Data Factory and Azure Databricks offer transformations at scale when it comes to ELT processing. On top of that, ADF allows you to orchestrate the whole solution in an easy way. In a case, when you prefer to use Scala, Python or SQL code in your process, rather than Mapping Data Flow in ADF - you must link ADF to Databricks. I will show you how to do that in two ways.

VIDEO INDEX:
00:00 - Intro
00:42 - Two methods
02:22 - DEMO: Connect via PAT (Personal Access Token)
08:14 - DEMO: Connect via MSI (Managed Identity Authentication)
14:42 - Summary

*** Hungry more? Let's connect. ***

#adf #databricks #datafactory

[claudioroberto]

Hi Kamil! I am start working with ADF and Databricks and your video help me understand the process of connection with linked service. I'm 🇧🇷 Data Engineer. Sorry my english. +1 Subscriber

[angelamessina]

Thank you this video was very helpful for my work today.

[ts]

the singlenode cluster how you configured it to access blob storage? I am getting error at fs.azure.account.key error

[himalikhasgiwale]

Thanks it was helpful Can you show how to connect azure db from databricks

[Marchelo]

Is there any way to share results between different databricks notebook activities in the same pipeline of azure data factory?

[yoshida]

Is there any way to link databricks with same permission in databricks? I just allow my A user to use A folder in databricks and just allow to access A folder when A user use data factory to create pipeline with databricks?

[pavithransivaraj]

Hi kamil I am facing an issue while creating DB linked service .
Error : User not authorised

Need your help or suggestions Like what type of access we need to create a linked service.

[iakshaykr]

Very good and straightforward video . Thanks Please provide your LinkedIn ID is possible, .