39. How to Capture Changed Data using Data flows in Azure Data Factory

In this video, I discussed about how to capture changed data using data flows in Azure Data Factory

Link for Azure Synapse Analytics Playlist:

Link for Azure Databricks Play list:

Link for Azure Functions Play list:

Link for Azure Basics Play list:

Link for Azure Data factory Play list:

Link for Azure Data Factory Real time Scenarios

Link for Azure LogicApps playlist

#Azure #AzureDatafactory #DataFactory

[kishoreevani]

thank you so much for sharing this. Please keep doing more on ADF.

[CoolGuy]

This is similar to 35th video of the same playlist. Please correct me if I am wrong.

[balajikomma]

Sir please make a video how much ADF knowledge is required/sufficient for power bi developer. Thank you sir

[rohitkumar-itqd]

How do we merge this changed data at the destination without having duplicates.. after the steps at this video??

[ViralDave_]

can we do the same in Azure synapse ? if yes please prepare a demo

[pratikpatel]

This will still run at the trigger so It can't be real time. isn't it right?

[shreyashchoudhary]

One scenario trick one : increment copy data from onprem db by checking we are only copying the data which is not previously copied

[antonyaswin]

I am implementing this, but will this be cost efficient for the system ? We know has to take only those delta data or the ones to be updated alone .. But does it have to select /scan the entire data in the beginning ?

[sakshiagarwal]

Hi sir what if the file has 1000 of data in both source 1 and source 2 i am getting fisrt 100 data with different set of data due to which it is showing mismatch for all

[shreeyashransubhe]

Thank you for great information. In an interview I faced one question: If there are 10 files in source and we use copy activity to copy files to destination and if 6th file fails, so on next pipeline run how can we start the copy activity from failed 6th file and skip all previous files till 5th. Thanks is advance.

[tejaswinibellam]

Is it possible to copy from postgres to postgres

[tejaswinibellam]

How to replace old data of table at si nk side with new data. Any replace function is there

[vigneshr]

Hi maheer,
I want to copy 100 files from a folder. But the pipeline fails after copying 50 files. Now I want to rerun the pipeline but I should only copy the 50 files which are not already copied. Can you help me with this functionality, I couldn't find this in your real time scenario playlist.
Thank u.