How to Install and Run PySpark in Jupyter Notebook on Windows

Hello world!

Hello World! Some of my students have been having a hard time with a couple of the steps involved with setting up PySpark from Chang Hsin Lee's excellent tutorial on "How to Install and Run PySpark in Jupyter Notebook on Windows for my Udemy course on PySpark for Data Scientists. So I went ahead and created a video filming myself doing the installation on my windows machine to help clear up a few questions or provide a little extra detailed where necessary.

I’ve provided a link to his article below as well a link to my course for anyone who is interested in enrolling.

Happy coding!

[muhamedademi]

THANK YOU THANK YOU THANK YOU, I have spent several weeks trying to install pyspark on jupyter and I could not I am very excited and with hope that the world has such good people, God reward you Only He can give you what you deserve.

[Loenbrad]

Hello there, wonderful video and thankful for Chang Lee as well. Here is an issue I ran into incase anyone encounters the same:

Followed everything the same Eden described! -- but because I already had Java installed didn't bother to do a fresh install. This lead to pyspark code running indefinitely on jupyter notebook without error messaging. My solution was to edit system environment variables -> environment variables... -> changed JAVA_HOME to C:\Program Files\Java\jdk-1.8
I do not know the true causation of this issue but my JAVA_HOME was set to a path for jdk-12 which clearly wasn't working for some reason. Best of luck gangsters.

[dennisni]

Best 17min on Youtube for installing Spark . I have gone through so many videos but none were able to solve the issues. Thank You very much!!!!

[rajeswaril]

Thanks a lot for this life saving video!! The most underrated video for pyspark installation. Wish all those who all are struggling to install pyspark watch this video

[rashadnelson]

Writing to you in the year 2023. Wow! Out of all the videos on how to implement PySpark with Jupyter Notebook, yours is the only one that works! Thanks so much! I will definitely purchase your course on Udemy to sharpen up my skills with PySpark. I have a data engineering project to complete that requires Apache Spark. This will definitely come in handy. Thanks so much again!

[pravhattripathi]

Been toggling here and there to find best content to install pyspark on jupyter notebook. Came across your content and this is the best I have seen. Thanks.

[jenithmehta]

After whole day of google, stackoverflow, I found your video. Wow, just amazing, thank you, now I have pyspark up and running. Thanks once again... :)

[ilichbriceno]

Lord Jesus, you are a great teacher Mrs Layla, thanks you so much, i've tried a lot of tutorials to run a code in Pyspark, no one it's interested in explain this step by step as you did, so i'm subscribing to your channel and your Udemy course. Thanks, Blessings!

[wetpenguin]

thank you, finally a good video that shows everything from 0 to 100 and it actually works.

[enesbesinci]

thank you very very much i was trying to install spark for almost 5 hours, finally i installed it in 20 minutes with this video.

[DileepKumar-kpxc]

Please don't skip the video by seeing the number of It is very helpful
Thank you so much Spent more than half day for solving the Finally watched your video... Got Sorted :)

[mikedeguzman]

Thank you, thank you, thank you!!! This is exactly what I needed. Spent hours googling the right method to use. Kudos to Mr. Chang for sharing his knowledge.

[mmnmjml]

Wanna thank you for such thorough guide! If only I've saw this article first, I could save hours of browsing for the solution

[imranyasin]

Excellent demonstration! Pyspark installed and ran smoothly!!

[suneethagopavaram]

Thanks a lot!!! I was struggling to install for hours and could do it in minutes with your video.

[ayansharma]

Thank you! this is the video that has the exact solution for what i was looking for. Thanks again. You Rock!!

[lukeshmannan]

Great video.. If anyone facing PY4JERROR, just install jdk 8, set JAVA_HOME= installation folder path and restart system

[anandhakumarcholendran]

Your are my saviour. 🤩🔥 Thanks for the tutorial 🙌

[prakashmudliyar]

Awesome...it's working on my Pycharm also by default :D

[taherparda]

Yes great video finally able to run pyspark on windows.
P.S: Use jdk 11, jdk 18 didn't work for me.