Azure Data Factory - Incremental Data Copy with Parameters

This Topic covers the following:
1. Incrementally Copy Data from one Database to Another
2. Parameters to Support - Can be run for multiple tables using the same pipeline

Other Videos on Azure:
Subscribe:

Azure Portal:

Storage Account:

SQL:

Data Factory:

[pushpakbansal]

Very good explanation, helped me in getting the job done.

[kolliuma]

Great Explanation Sir with good content

[chandrasekharnallam]

nice explanation with easily understanding the concept. thank you

[shaileshsondawale]

Awesome Content.. Great work done Sir....

[Lihka]

@
Azure Bits and Bytes
Hi Sir very useful and excellent, could you please add update logic, so if there is any change in source table records it should update the records in sink table then need to take latest from source to sink table. Thanks in advance

[juzerpatanwala]

where can we find this dataset of customer orders

[ashwinds]

Am getting an error in pulling the data through ADF for Oracle as a Source . Could you please help me in this

[vishwanathr]

Hi

How to Filter multiple values in Parameter like

SELECT * FROM DimCustomer WHERE EnglishEducation IN
not sure it is correct way to pass multiple values in to parameter in adf paramter.

Thank you

[preetijaiswal]

Hi, This Really worked and helped me out . Now I need to do CDC based on other column like Id or other unique column so how to implement this as our watermarkcolumn is of datetime type.
Please Help me out.
Thanks in advance

[krishnakumar-vitf]

It was nice, but one question how can i handle multiple tables passing dynamically to pipe line, right now you are entering table names at pipe line level manually but my requirement is i have to pass table list for both lookups dynamically

[dataengineerazure]

PLEASE HOW GET FILE SOURCE FOR TRAINING? TKS

[hungerspoint]

How it will handle duplicate data?? If pipeline runs twice at at time the. It will again copy the data??

[nagarajupinnu]

why can't we update watermarkDate in a table directly with query instead of Stored procedure which is extra object

[wheretastesdivedeep]

How to handle if the table don't have data?

[rohitsethi]

no this is not useful.The table should be manually enter. This is 200% end developer can end wrong table or misspell or space in table name end or before. The best way to store all the table name in the Table then for each loop process