Tutorial 5- Pyspark With Python-GroupBy And Aggregate Functions

In this video we are going to discuss about groupby and aggregate function using Pyspark

Subscribe my vlogging channel
Please donate if you want to support the channel through GPay UPID,

Please join as a member in my channel to get additional benefits like materials in Data Science, live streaming for Members and many more

Connect with me here:

[anandvardhan]

Enjoying your videos. I was puzzled with all the stuffs out there on the internet. Lucky to find your channel which is a full package in itself.❤️

[karthikkarthik-lswg]

Really nice and inspiring sir for all your work and knowledge 👍

[ajaykiranchundi]

Appreciate your crisp and neat explanation.

[jithmiranatunga]

Most awaited seiries❤Thanks a lot sir.

[manuize]

Sir my question is
by creating you can run sql query like spark.sql('select * from temp_table') why we need to learn pyspark syntax .

[ashishbhatnagar]

amazing content. Very very useful for us. Keep continuing this series. Again thanks a lot for such amazing content

[UmerPKgrw]

No example of allias. No example of applying the different aggerate operation on different columns For Example Average salary against each deprt and Count on number of people in each dept.

[rohitjagdale]

Thank you for the video 👍🏽 Can you please make video on how to use koalas and pyspark ?

[vinayakdhruv]

Hi sir, been following your videos and articles for a while, and it has helped me a lot you are an amazing teacher and guide.
I do have a query regarding data science career options, where can I get in touch with you sir?

[riajain]

Hi Sir
How much time will it take for you to upload entire pyspark series?!
Also, thanks for the amazing content.

[subhajitdey]

Thank you for this video.
I have one question. Is it possible to get all columns when using groupBy()?

[rohitjagdale]

Hi Krish, I have used koalas with PySpark 3.1.1 on Google colab. I am getting error while using "figsize" in plot method of Koala dataframe. All other code is working fine. Can you please help me with "how to set figsize while using plot method of Databricks Koala dataframe". I am using latest version of koalas and plotly.

[chillagundlavamshi]

I get this error by using groupBy function and solve this error
Analysis Exception: Column 'Name' does not exist. Did you mean one of the following? [Name, Salary, Department];

'Aggregate ['Name], ['Name, sum(Salary#19) AS sum(Salary)#152L]
+- Relation [Name #17, Department#18, Salary#19] csv

[naveenkrishnan]

Sir this time deep learning course in ineuron is available or not? Sir

[pavanjoshi]

Hi Sir
whenever i am reading .csv file by using df_pyspark.show(), I cannot able to see data more than 8 rows, Can u plz help

[sarthaksarjine]

i want to get max 5 values from a particular column of dataset. how should i do it by using max function?

[papachoudhary]

Thanks! Sir, how should get complete session of these . What is fees and cintact number or links to connect with you

[amitbudhiraja]

Hi everyone,
When i am pressing tab after this to see the more functions i am not getting the results instead i am getting this ipynb_checkpoints/ again and again .
Now how to see the more functions then ?

[jikkuization]

why no videos about RDDs which is the main datastructure in pyspark

[zohaibramzan]

One thing is still not explained: how to add new coloumn to pyspark dataframe if column is a list. Kindly explain this one too ?