Spark Dataframes vs SparkSQL

preview_player
Показать описание
What should you use, Apache Spark Dataframes vs Spark SQL?
Рекомендации по теме
Комментарии
Автор

My first job we wrote our rickety ETL's with pandas. God forbid if we were ever to get more business, all the pipelines I wrote would have failed due to out of memory issues. I've been avoiding spark as much as I can but I gotta admit the spark ecosystem is very rich.

andynelson