Webinar Recording: Incremental Load with SSIS

Title

Incremental Load with SSIS

Description

Incremental Load is always a big challenge in Data Warehouse and ETL implementation. In enterprise world you face millions, billions and even more of records in fact tables. It won’t be a practical practice to load those records every night, as it would have many downsides such as;

ETL process will slow down significantly, and can’t be scheduled to run on small periods.
Performance of the source, and destination server will be affected badly, downtime of these systems would be longer.
More resources will be required to maintain the process. such as better processors, more RAMs… and adding these won’t help so much at the end, because the amount of data is increasing as times passes.
and many other issues.

So what would be the solution? Solution is Incremental Load approach. In this approach data will be loaded partially, preferably only part of the data that has been changed. A change set will be much smaller than the total amount of data. As an example in a 200 million records fact table which stored data for 10 years, only 10% percent of that data might be related to the current year and changes frequently, so you won’t usually required to re-load the rest 180 million records.
In this session I will show you how to implement Incremental load with SSIS with demos.

Speaker:
Reza Rad is an Author, Trainer, Speaker and DW/BI Consultant. He has a BSc in Computer engineering; he has more than 15 years’ experience in databases, programming and development mostly on Microsoft technologies. He is a Microsoft MVP in SQL Server for six continues years (from 2011 till now) for his dedication in Microsoft BI.

He wrote some books on MS SQL BI and also is writing some others, He is a International Speaker in SQL Saturday and SQL users groups and Conferences like TechEd and PASS Summit. He is a Microsoft Certified Trainer.

[arjunsurendra]

An change data capture be used for near to real time data warehouse incremental loading ?

[prathmanu]

Very good tutorial.But what if we need to do it for 100 databases with each db having 300 tables?