Azure Data Factory | Deployment from master branch code (JSON files)

In the previous episode, I showed how to deploy Azure Data Factory in a way recommended by Microsoft, which is deployment from adf_publish branch from ARM template.
However, there is another way to build CD process for ADF, directly from JSON files which represent all Data Factory objects.
In this video, I will show you the latter method, building the entire process step-by-step, including the fact that we have multiple environments and wherefore isolated services. Therefore, we need to manage all the needed properties, correctly.

This page is dedicated to one topic only: Publish your ADF from code to the service in Azure!

Update @ 29/11/2020:
This video has been cut off now as the whole process of ADF deployment presented here is too complex. I'd like to redirect you to two other videos on my channel.
The only reason why I didn't delete the video entirely is to keep all comments below.

Update @ 02/06/2020:
Since now, I recommend this extension to Azure DevOps only if you plan to deploy ADF from code:
It is free from objects dependencies issue and does all the work in one task. Check it out.

Update @ 02/05/2020:
I have spent several days and built a brand new PowerShell open-source module to publish the whole Azure Data Factory code from your [master] branch or directly from your local machine. The module resolves all pains existed so far in any other solution, including:
deployment part of objects, deleting objects not existing in the source any longer, stop/start triggers, etc.
More details and full documentation with code is in GitHub:

*** Hungry more? Let's connect. ***

#devops #cicd #json #adf #azuredatafactory

[shaxxie]

Your video at 7:34 just completely jumps to another topic and misses the main bit of inserting the tasks into the release pipeline. 😢

[deoyanidhote]

great video ..thanks for sharing it✌.. Can we change adf pipeline parameters as well with this method?

[juanfelipeperezramirez]

does this video really last ~9mins? I think it was cut. Is there anyway to have it complete again?

[ritwikc]

Very well explained video. Could you answer my query: Instead of using the Azure Data Factory Deployment agent, is there any way to deploy using the Microsoft recommendations. As specified in your previous video I tried to deploy using ARM templates. But the linked resources were pointing to the DEV infra as shown in this video. So is there anyway to use the ARM template deployment with linked services of QA values

[vasuarjula]

Kamil, Great explanation on deployment from adf_publish and from a master branch directly. We are using Azure DevOps GIT and is there is any way to deploy individual objects rather than an entire branch. I just wanted to deploy 2 or 3 pipelines out of 20 changed pipelines from DevOps GIT. Is this possible?

[SantoshSingh-kibx]

Kamil, Just want to confirm here that do we have keep separate CSV(config-UAT.csv) files for each stages in deployment folder. Let say I have two stages preprod and prod in my DevOps project. else how to manages using single csv file in both stages Ci/Cd environment ?

[pratheeshprabhakar]

Do you have a similar solution for synapse pipelines?