Pyspark error Could not serialize object | How to solve

preview_player
Показать описание
In pyspark, if u get Could not serialize object error how to solve? explained in thsi video. Main problem it's Python version issue. must use these version only
Spark 3.1.2, java 11, python 3.8, hadoop 3.2.2 and scala 2.11 version only.

for more tips subscribe
for spark training pls call:+91-8500002025
error:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
_pickle.PicklingError: Could not serialize object: IndexError: tuple index out of range
Рекомендации по теме
Комментарии
Автор

I did downgraded to 3.8, nothing resolved, still same error

shelly
Автор

idiotic "solution"...
According to spark3.4, it should work on python3.11

PizdaRusni