Azure Data Factory - Incremental Load or Delta Load for Multiple SQL Tables in ADF

Azure Data Factory - Copy multiple SQL tables incrementally using a watermark table (Delta Load) ADF

Here we are doing incremental copy of multiple SQL Tables from one Database to another Database.
Please refer the below video Step by Step explanation of pipeline for a Single SQL Database:

We connected to SQL Database from On-Premise Environment for this video. To know how to setup Self Hosted Integrated Runtime,
Please refer:

Check out my Udemy Course on Building an end to end project on Azure Data Factory and Azure Synapse Analytics

Udemy course with Coupon link :

My New course on Complete Azure Synapse Analytics:

Limited time to enrol this course with above Link for can access the course with 50% OFF!!!

============================

============================

[mro]

What if I can't modify the original schema adding a table and create a procedure
Can I use a flat file in Azure to store timestamps ?
Following this idea, how to write tablename and new timestamps values into a new file and replace the old file by the new one ?

[bulubrr]

Hi @Shanmukh Sattiraju can you please send me the stored procedure for the water mark table updation for multiple tables

[somesulasankar]

Hi Shanmuk
regarding ADF how can i reached out

[emhbnwx]

hi, i am new to using data factory and learning about the ETL process. i want to be able to keep the information in a table updated and synchronized every day, would what you do in the video work for that? or do i need a control table?

[souranwaris]

I need your help, I am stuck for many days at one point.

could you tell me how To load data from an on-prem SQL Server to Azure Blob Storage with delta load or Incremental load?

I have a problem with the delta load I want to create an incremental pipeline from the Oracle on-premise server to Azure data lake(blob storage) I don't have Azure SQL. I just want to push in blob storage as a CSV file. in my case, I have confusion about where I should create the watermark table. someone told me in your case you have to use parquet data. please help me with this.

Thanks

[viniciusandrade]

Hi, Is there any way to copy data from multiple sql tables to blob storage creating folders by data ( year > month > data ) ?

[ranjansrivastava]

Dear, how are you handling the incremental load, is there upsert stored procedure written or what is the logic kindly share.

[anushachundru]

I am getting an error while publishing the pipeline as "No value provided for paramater TableName" from Look up source actvity

[rohitkulkarni]

I need to understand what all the values is mentioned in the both locktables

[RajendraKothapally]

your concept is interesting, but your voice is very low. Its hard to hear.