Going From Data Engineer To Head Of Data - How To Run A Data Team Successfully

You join a 1000 person company as the head of data. What should you do?

I would invest a a lot of time up front to understand the business(especially if you haven't worked in the industry).

I was just talking to someone at the Snowflake Summit who told me they made the mistake of recently being put in charge of a data team and their first response was "Great, what tools can I use".

If you can't answer the following questions within the first month or two, you're probably going in the wrong direction.

- What are the main drivers of the business?
- What are the main pain points of the customers?
- What does the business flow look like?
- Who is our customer?
- (What other questions would you add)

More than likely, if you're a Head Of Data, you've hopefully got the technical chops to excel in the role.

But if you don't have a strong understanding of the business, you'll either end up becoming a task taker who struggles to get out of ad-hoc requests or you'll build a bunch of fancy infrastructure that delivers nothing for the business.

If you enjoyed this video, check out some of my other top videos.

Top Courses To Become A Data Engineer In 2022

What Is The Modern Data Stack - Intro To Data Infrastructure Part 1

If you would like to learn more about data engineering, then check out Googles GCP certificate

If you'd like to read up on my updates about the data field, then you can sign up for our newsletter here.

Or check out my blog

And if you want to support the channel, then you can become a paid member of my newsletter

Tags: Data engineering projects, Data engineer project ideas, data project sources, data analytics project sources, data project portfolio

_____________________________________________________________
_____________________________________________________________
About me:
I have spent my career focused on all forms of data. I have focused on developing algorithms to detect fraud, reduce patient readmission and redesign insurance provider policy to help reduce the overall cost of healthcare. I have also helped develop analytics for marketing and IT operations in order to optimize limited resources such as employees and budget. I privately consult on data science and engineering problems both solo as well as with a company called Acheron Analytics. I have experience both working hands-on with technical problems as well as helping leadership teams develop strategies to maximize their data.

*I do participate in affiliate programs, if a link has an "*" by it, then I may receive a small portion of the proceeds at no extra cost to you.

[Ian-vhvv]

Just went thru this process with my company the past year. Great video. With us it went something like:

- Where is all of our data
- How are we doing reporting now
- What are the shortcomings of existing reporting solutions
- Do we need a warehouse (yes)
- What warehouse do we pick
- What ETL stack makes sense for our use case
- What do we integrate in what order to maximize value and get adoption rolling

Also, Having someone on the exec level champion the BI effort and really push it forward was huge for the thing to actually materialize.

[smrtysam]

This has happened to me. Now I’m leading a team of data scientists, engineers, analysts and migration specialists. I’ve had to learn so much so quick about strategy and people management. I’ve had to coach the people on my team to really empower and own their own tasks. At the beginning of being head of data I was taking on way too many “low level tasks”. Now I’m delegating and empowering. I still have alot to learn though.

[William-B]

We’re a young data team for a large organization. Biggest roadblocks for us are issues with data governance (“you can’t have or report on our data”), budget for tooling (“prove the value of the tool, then we can purchase it”), and cloud concerns (“all my data is on-prem. You can’t just put it in the cloud”)

[CowboyHabibi]

Cool channel! I noticed that Snowflake's stock has been pretty bombed out since IPO. Are they still as integral as they were when they went public?

[burnier]

I have 20+ years in the business and would never become head of data or manager... I'd rather keep doing tech stuff and have as few human interactions as possible 😅

[sirus]

I keep hearing from top CEOs that with Palantir we don't need teams anymore

[crazyphantom]

I wonder how much Data God has in the Aether/Astral realm 🤔