WHAT Is 'Pickle' In Python?! (EXTREMELY Useful!)

What is "Pickle" in Python? Well, pickle is a module that allows us to serialise and deserialise objects in Python. So that we can save some more complex/ custom data types such as instances from our classes!

▶ Become job-ready with Python:

▶ Follow me on Instagram:

[evlezzz]

You went to a very dangerous territory here...

Using pickle to store state of objects in long term has a lot more caveats than just "don't load random stuff from the internet'.
1. Pickle is Python only format. You can't read it from anything else or look at stored data. Even within python there are several versions of pickle and you can't load files stored in newest pickle format on older versions of python. (Last change happened at python 3.8)
2. More importantly, pickle depends deeply on the structure of your code, modules and classes. You can only safely unpickle objects that were stored with the same code structure. Programs tends to change over time. Storage formats change less often and it's easier to consider what to do with old data. Add new attribute to your class or rename one. If you attempt to load pickle file stored in previous version, an object will be restored with attributes as they were in a previous version and they probably won't match the changes you made. Rename the class or put it into different module — and loading will fail.
3. There is a reason why saving into json takes a lot of repeating code: you actually need to say explicitly, what are you planning to store and what name should be attached to it. Your classes could contain some data that is deduced on the fly and there is no use in storing and loading them. And without additional code pickle would just store everything from object (well, technically not everything, there are some exceptions by default). You could specify what to pickle and what not to pickle, but then we return to the same state as before which is — we need to be explicit about what to store and how.
4. There are proper ways to stay with normal interchangeable formats and not writing too much boilerplate. Projects like pydantic, dataclass-json, dataclass_wizard, dataclass_factory and others allows you to specify mapping between most of formats (like json, yaml, or some dictionary loaded from sql database) and your classes.
5. Main purpose of pickle remains to be short term storage of objects so they could be transferred between parts of the system that you maintain, for example, for interprocess communication or sending tasks to worker farm run by celery. And pickle was never intended for long term storage of the system state like you are implying.

To summarise: don't use pickle this way if that's not a one-time task and you are planning to use what you have written two month later, there are better solutions.

[catloafl]

This is pretty awesome; I will definitely use this in my future projects! Thanks for the tutorial 😄😄

[LukeMartinVideo]

Lovely quick and easy lesson. Love it. Very instructive too !

[ddesy]

Cool video, thanks for the disclaimer at the end, these types of videos rarely mention the downsides of certain methods

[ashersaipe]

wow this is so useful thank you! nice video as well, very clear :)

[danielniels]

thanks! really useful for me as a Python beginner. Subscribed!

[anoops]

Thank you so much. Your teaching is very concise and I was able to understand it super easily.

[barelycodingtoday]

I used this today on a Project. Thanks!!

[NewsSphereNetwork]

This is a good one, Didn't know the real usage of pickle before, Tnx

[miguelvasquez]

great video, like the ones you have done before

[adrijeguha]

What's the difference between object creation like the one shown in the video and `fruit = Fruit('Banana', 100)`.

[oshriperetz]

For shorter, more concise and safer code, look into dataclasses and the "dacite" package :)

[JustComputers-qnet]

" with open('banana.json', 'w') as file:
json.dump(fruit.__dict__, file) "

&

" with open('banana.json', 'r', encoding='utf-8') as file:
fruit = Fruit(**(json.load(file)))"

Does this not achieve the same thing but for json? rather than a need to manually create the data you could dump the object as a dictionary?

[chikezieezenna]

I will know the usefulness of pickling a model for example a machine learning model you have trained.

[riyalmasla]

Check out dill if you want to export objects as files.
dill was a very useful module for me.

[yusinwu]

Will this work if the python script loading the pickle file doesn't have access to the class definition?

[jma]

pickling isnt safe so it should only be used on cases were you trust the data, like cache or something

[RetroGamingRandomness]

Id personally use it on a vm or public computer if it was a random pickle file

[marksegall]

'data.bin' is a typical file name for the pickle module. Pickle creates binary files.

[chrisnorthall]

doing my CS MSc using pickle is a big no no, assignment gradings are penalised for doing so. I can see the appeal...