30 How to do incremental load in SQL server

how to do incremental load in sql server
Download the file\script used in the Video from below link

how to do incremental load in sql server
How do you do incremental loads?
How do you do incremental load in SSIS?
What is incremental refresh in SQL Server?
What is incremental load in ETL?

[shaikaniph]

Very good video for incremental load using sp. This is exactly how we do in real-time environment.

[eltonmelo]

Ótimo vídeo, obrigado por compartilhar esse conhecimento!

[dfkbthv]

Excellent explanation, thanks Aqil sir

[deepakj]

Great efforts i appreciate it.. Thank you Aqil !!
hashbytes() function is really useful.. Pls add new concepts like this.

[leemabandelu]

I'm updating only a single column value, in th source table, and executing the. Sp but it shows more than 1 row affected... Why is that ? I wrote stored procedure exactly like yours.

[mocanada]

Hello Aqil (The wise! :))
Could you please also add the delete functionality in the solution? Or you can make another video showing that?
What if you are transferring a transactional table and the data is deleted in the source but it is still in the destination?
It would be nice to have your data fully synchronized with the destination.
You are a gem!
Can't thank you enough!
Thanks again from Canada!

P.S. Maybe capture the deleted records information in a log table? So that we can investigate?

[abhilashpatil]

Hi, When I run a job from the SQL Server agent I get a step1 error kindly help my source is an Excel file I have created an SSIS package in visual studio if run the package from visual studio it will run successfully but the same package if I run from SQL Server agent, I am getting a step 1 error.

[pratikghimire]

This is great video, Please could you also make on SCD Type 2 using SP. I believe, this must be similar process. Thanks a lot

[IngenieroAguilera]

Hi, thanks for this video. I have some questions:
Why did you use hash bytes function?
Was not it enough by using concatenation and comparing?
Could this hash bytes function deteriorate query’s performance?


Thank you!! 🙌🏻

[divyasri]

Great video 👍.I have a question suppose we have 2 or more sources then how do we need to process the data

[mocanada]

Hello Aqil,
Your parents named your correctly. Aqil means knowledgeable. Indeed you are very knowledgeable.
Thank you for all these amazing videos.

I have 4 questions, if you could answer it, it will greatly appreciated.

1. Is there away you could make the column names get populated dynamically? Like if I want to perform this task for over 100 tables, then it will be too much work to manually type all the columns.

2. Will this work if you have tables that have almost 100 million records?

3. Does this work for records that get deleted in the source?

4. Is there I can find out which records were added or updated?

Thank you so much!!!!

[noopurdubey]

Thank you very much for the video. Is it possible to run this stored procedure automatically whenever there are changes made to source table ? or may be we have this stored procedure triggered using any Google cloud service like cloud function etc ?

[catulopsae]

Great video. I have an issue when downloading from Snowflake into SQL server some columns are being downloaded with wrong data. What provider should we use to connect in Ssis and sql server?

[yathishkumar]

Hi, i would like to know the procedure to load a date column that having a 100 dates with two different formats(7/31/2012,
02-07-2013) in to SQL. i have tried a lot but didn't workout. could you please provide the solution.

[juanodonnell]

why do you compare the hash columns and not compare a simple string concatenation?

[kaiyao]

hi, Thanks for your video! just wonder what if the tables are in different database or different instances?

[leemabandelu]

Also hashbytes can be used for varchaf datatyle, what about int datatype ?

[tvenkateswarlu]

HI Aqil, How can we find last updated record in destination table when we dont have any id and Timestamps.

[vinayatavakari]

..I need to store output of query in same database is it possible?
Data should save in incremental
After every run

[JeetShreeOfficial]

Thanks,
Q:- I can delete records using your code also but it is not capturing by Audit/log table, please help me out to track deleted records in Audit table