PySpark Tutorial 2: Create SparkSession in PySpark | PySpark with Python

PySpark Tutorial 2: Create SparkSession in PySpark | PySpark with Python

About this video: In this video, you will learn how to about Create SparkSession in PySpark in pyspark

Large Language Model (LLM) - LangChain

Large Language Model (LLM) - LlamaIndex

Machine Learning Model Deployment

Spark with Python (PySpark)

Data Preprocessing (scikit-learn)

Social Media Links

#llamaindex #openai #llm #ai #huggingface #api #genai #generativeai #statswire #spark #pyspark #python #pythonprogramming #pythontutorial

[arihantjain]

Can you help me my spark session is taking forever to create like 1 hour the command is still not executed. What to do?

[Sehajbir_Singh]

is there a way to save the session before exitingg, and open and continue from where we left ?

[satishmajji]

You said both are entry points for spark functionality. What's the exact difference between SparkContext and SparkSession?

[parayou]

I am trying everything, but I can't initialize the Spark Session

I got stuck on this Error:

NameError: name 'SparkSession' is not defined

Could you help me ?

[glong]

Hi. I had followed your instruction but it got an issue: my Executors in Spark UI shows nothing. Do you have any idea?
Thanks for your helpful video.

[ashwinjoshi]

Hi,
I am facing below issue while importing spark session
Error : "cannot import name 'SparkSesson' from 'pyspark.sql'

I used the command
"from pyspark.sql import SparkSesson"
I am not sure if it is anything related to path ?

[_venkateshtohgaonkar]

hii i am facing issue while creating the spark session
ConnectionRefusedError: [WinError 10061] No connection could be made because the target machine actively refused it

[helovesdata]

How do you setup the 8 cores on a local system? I put local 4 and its only using 2.

[nullb]

When I execute line 2 I get: "The system cannot find the given path". in cmd
Do you know how I can solve that problem?

[shivakant]

I launched spark shell but sc or spark is not available please solve this issue.

[anupgupta]

I sm always getting error this spark may be existing one but I am trying for first time

[لاالہالااللہ-عوز]

I am stuck here " he is showing error : AttributeError: 'Builder' object has no attribute 'getOrCreat'