Snowflake Vs Databricks - 🏃‍♂️ A Race To Build THE Cloud Data Platform 🏃‍♂️

Databricks vs Snowflake

Two data storage solutions that started in very different worlds converging on the data platform.

Both want to be your one stop shop.

Your data warehouse and data lake

Your data lakehouse...

But really they want to be your data operating system.

If you're trying to pick between Databricks And Snowflake, or perhaps another data storage solution, then we should talk! Set up a free consultation today

Looking to start you're own data engineering/analytics consulting company, then you should check out my new course here

Also if you enjoyed this video, check out some of my other top videos.

Top Courses To Become A Data Engineer In 2022

What Is The Modern Data Stack - Intro To Data Infrastructure Part 1

If you'd like to read up on my updates about the data field, then you can sign up for our newsletter here.

Or check out my blog

Articles from snowflake and databricks

If you would like to learn more about data engineering, then check out Googles GCP certificate

And if you want to support the channel, then you can become a paid member of my newsletter

Tags: Data engineering projects, Data engineer project ideas, data project sources, data analytics project sources, data project portfolio

_____________________________________________________________
_____________________________________________________________
About me:
I have spent my career focused on all forms of data. I have focused on developing algorithms to detect fraud, reduce patient readmission and redesign insurance provider policy to help reduce the overall cost of healthcare. I have also helped develop analytics for marketing and IT operations in order to optimize limited resources such as employees and budget. I privately consult on data science and engineering problems both solo as well as with a company called Acheron Analytics. I have experience both working hands-on with technical problems as well as helping leadership teams develop strategies to maximize their data.

*I do participate in affiliate programs, if a link has an "*" by it, then I may receive a small portion of the proceeds at no extra cost to you.

[markwan]

Love this. I'm a Solutions Architect and the information you give out is priceless and accurate. Well done!

[brothermalcolm]

I must be watching too many of your videos lately - searching for databricks landed me almost straight here

[mg]

Micropartitions is just another name for... partitions aka shards, something that all databases use to scale out (because that's the only way to do it). There have been dozens of databases doing this for data warehouse/OLAP uses for decades. Snowflake (like BigQuery) was more powerful because of cloud-native scaling rather than provisioning real hardware, not just separating storage and compute. The other thing was usability from being in the cloud and running on object stores with features like zero-copy clones, sharing datasets across companies, etc.

[permiek]

Spark (and therefore Databricks) is really a game changer

[kevintaylor]

Snowflake is for BI and more traditional analytics. Excels in data warehousing, storage, analytics.
Data Warehouse engineers and Data Analysts.

Databricks is for big data processing (machine learning, AI workloads).
Data Engineers & Data Scientists.

[thomsondcruz]

Snowflake provides credit usage per second for each T-shirt size no VM costs to run the service. Databricks has its DBU cost per second for it’s service plus there is an underlying cloud VM cost to run databricks. This could be a plus or minus because databricks allows you to select the type of VM you want - Compute optimised vs Memory Optimised

[alexanderpotts]

coming from an SSIS background I figured I'd end up landing in snowflake when I got to working with cloud tech, but I've really, really taken to spark and databricks. even if writing pipeline code in notebooks scares me sometimes

[cyclonus]

Thanks for the overview. Our organization went through a very contentious evaluation internally between these two platforms. Ultimately ended up going with SnowFlake due to internal politics. Would also love to see a video comparing all four: bigquery redshift snowflake databricks

[agilejro]

I feel Streamlit has helped increase the depth and specialty application possibilities for Snowflake while keeping the streamline usability, do you feel this is true?

[andrew]

our boy has been hitting the gym.
loved the philosophical background on them at the beginning.

[eklok]

Awesome video and comparison! Was missing some details on their features in the sense of functionality e.g. a Delta lake might have time travel while snowflake has some pretty Advanced sql functionality that one might use for gdpr hashing. On these „implementation“ details i Would love to see some content. Also bq vs snowflake would be very interesting i guess

[riffsoffov]

Thanks for the video. Also thanks for the relevant title, rather than clickbait or one tailored for youtube's algos. I found you because Codestrap mentioned you (youtube's algos are useless for finding people who know what they're talking about).

[pavankumar-nimy]

I primarily do ETL activities for a product based org and would like to know when to use Databricks and Snowflake or rather how DB/Snowflake can help ETL engineers who migrate data from one product to another.

I couldn't understand this video properly because I was expecting a one line answer on when one should use Databricks/Snowflake

[digithat]

Been loving snowpark sno far. It's like a wrapper of spark but by snowflake.

[rajandalawai]

I suggest that topics such as this that are heavy on content, it is better to go a little slower

[zvumpbw]

You missed core difference's. Databricks has a strong flexible offering that includes, ml, geospatial, ect. Serverless, ephemeral clusters ect.. snowflake does hosted data warehouse well... Thats what makes it good. You don't want to mess with managing complexity use snowflake ( with databricks) if not use databricks

[seasql]

need both databricks and snowflake for any enterprise application.. databricks ingestion the data from various sources and processing the data and stores in any database such as snowflake and postgres... snowflake is mainly used to OLAP application.. we cannot stores the data in delta table in databricka.. it's taking more time to return the data from delta table due to the more processing steps involved in spark.. job, stages, task, partition and cores but these kind of processing steps is not involved in snowflake and snowflake return data at earliest..

[nicky_rads]

Nice video detailing two very popular data products, thanks !
Personally a big fan of notebook dev, but as you explain both products are solid and can serve customers well.

[SreejaThumma]

Can you also make a video on the difference between DataBricks, Snowflake and Solix technologies

[janardhand]

Big like from india.. nice presentation