Complete Machine Learning Project with PySpark MLlib Tutorial ❌Logistic Regression with Spark MLlib

Microsoft Azure Certified:

Databricks Certified:

---

---

COURSERA SPECIALIZATIONS:

COURSES:

LEARN PYTHON:

LEARN SQL:

LEARN STATISTICS:

LEARN MACHINE LEARNING:

---

For business enquiries please connect with me on LinkedIn or book a call:

Disclaimer: I may earn a commission if you decide to use the links above. Thank you for supporting the channel!

#DecisionForest

[DecisionForest]

Hi there! If you want to stay up to date with the latest machine learning and big data analysis tutorials please subscribe here:
Also drop your ideas for future videos, let us know what topics you're interested in! 👇🏻

[LuciaBukovaLushspaces]

Great that you've put the timelines in the description. Really helpful!

[hilmi]

Hey, Radu! Thank you for these very informative and practical tutorials. They really helped me to figure out big data preprocessing and building pipelines. Please keep going and keep adding new tutorials.

[amitkumargangwar]

Hi there,
I think have made a basic mistake, you are actually using test data (pp_df) to train the model.

[rezahamzeh]

Missed your amazing training videos. Hope to see more of them in near future

[TheLeoncer]

Liked and sub'd. Incredible. I pay my lecturer thousands of dollars and he can't explain nearly as clearly what you have just showcased.

[DataScienceGarage]

Very rich explanation, thanks for that!

[saurabrao]

It would really help if you can prepare videos for end to end pipeline and production based implementations of other ML algorithms using Mlib, like RandomForest, LinearReg, SVMs etc. Thank you!

[haneulkim]

Great tutorial! What if you want to preprocess using pyspark but then convert back to pandas for use of tensorflow? In such case I would need to extend onehot encoded vector into separate columns with correct column names, Any tips on doing this?

[fatenlouati]

Thank you for this lessoon. I wish you explain how to do this in a spark cluster

[shann]

I was so helpful, it will help to make a homework, thanks

[amitsrivastava]

Hello There!i am getting the error message at onehotencoder steps such as "TypeError: __init__() got an unexpected keyword argument 'inputCols".can you please help

[guneetkaur]

__init__() got an unexpected keyword argument 'inputCols' getting this error in the step one_hot_encoder = [OneHotEncoder(inputCols=[f" {x}_StringIndexer" for x in catCols],
outputCols=[f" {x}_OneHotEncoder" for x in catCols])] PLEASE HELP!

[tatidutra]

Thank you for this video! It helped me a lot! :)

[carlvinabonyo]

Great video subscribed!, would you know or have a video of, if pyspark ML has a functionality similar to Voting Regressor in sckit where you can use multiple regression types like linear, random forest and create one ensemble that takes the average of the predictions of these multiple models.

[pronoy]

I am using PySpark MLIB for multiclass image classification. Can anyone suggest to me the stages of my deep learning pipeline for the concerned task? I am using the latest PySpark version so things like DeepImageFeaturizer have long been deprecated

[alejandrofleitas]

Excelent video congrats. One question, whats the diference between Spark mlib and pyspark mlib? Thanks!

[flamboyantperson]

Thank you very much for this topic. I loved it.

[mayraju.p]

Hii, can you please explain to me the last part of the recall and precision table and how can I understand that one?

model.summary.pr.show()

[flamboyantperson]

Also make one on k means clustering algorithm with pyspark