Implementing SCD Type 2 using Delta

This video shows how to implement SCD type 2 using Delta tables. This is similar to the method available in SQL.

[omkarshirsat]

can we have a some logic where we do not have to use this mergekey as extra column, as I also want schema evolution along with scd2, but using this approach with schema evolution I am also getting the mergekey column in the output, which I don't want, so is there any way?

[chintusingh]

Merge will not work if there are duplicates in the source. Is there any workaround for the same?

[lokeswarv]

Hey, Try to run merge query again and again. It will insert the records into Dim Table.. Beacuse of joinkey considering as null always EmoloyeeId from target not matched with null and keep on inserting records

[SatishKumar-ntt]

if there is no change in source data and we try to run the merge code again as part of daily run then the mergeKey null records will be inserted again into target column as active and we will be ending with duplicates, how to solve it ?

[sreejithm]

Good video. what does the 'Using Delta' in table creation statement mean?

[kenpachi-zaraki]

can you please write scd type 2 code in generic way currently you have written it only for the one column please and thank you.

[tud]

good ...but what about the constraints u didn't use any constraints

[vinayakbiju]

can you share the file you have uploaded here
the csv file

[rajsekhargada]

what if othet column updated apart from address

[piotri]

Aren't you updating ALL historical versions of given customer instead of just the current one? Let's say you get new address of customer A od day 1 and the next day another update of the same customer. In the latter case you should be updating the end date of only the version from day 1.