10 Years Of Building Data Pipelines - What Has Changed And What Has Stayed The Same

preview_player
Показать описание
When I first started in the data world, the first tool I used to build a data pipeline was SSIS.

Since then it feels like I have come across every possible tool and custom data pipeline set-up possible(of course thats far from the truth).

There seem to be hundreds of tools and methods for data teams use to get data from point A to point B.

So I wanted to share some of those experiences as well as hear from Daniel Palma's experiences building data pipelines.

What has changed?

What has stayed the same?

What challenges do data engineers still face today?

Feel free to share some of your questions below!

Рекомендации по теме
Комментарии
Автор

I started with scheduled queries and apis, currently working in data governance, I do love how much more automated things are now, thinking of getting back into data engineering.

MrThedukekid