BINARY vs TEXT File Serialization

💰 Links to stuff I use:

This video is sponsored by Brilliant.

[LordHonkInc]

Ah, that brings back the nostalgia of opening an .ini file of an old game for the first time and changing things just to see that it actually changes what the game does. It broke things more often than not, but that was my first interaction with the concept of variables.

[craftydoeseverything]

Serialisation is a topic I've wanted to see a video on for so long, I got so so excited when I saw this video was released!

[anon_y_mousse]

One thing to remember is that when using "plain" text that the encoding and file formats still matter quite a bit too. For instance, whether you use ASCII or UTF-8 can make a huge difference, and while YAML is an okay choice, TOML would be better because whitespace really shouldn't be significant in a user edited format.

[cnb_kevin]

The best solution is USE BOTH.

On my current project(s), I use only binary because they are small projects.

If I had more time, the level files for example would be saved as xml or json. That way merging is easier in git, and if I break loading/saving code and save a broken file, I can edit the broken data manually. Then I'd write an exporter to export those text/json/xml into binary when I export the project data. Only the problem now becomes that you need an additional QA pass to check that binary data also works.

Although, player options are good to export as xml, so users can change things. Which is what I used on my previous project.

And if you're going to use binary, as ThePrimeagen said recently on his stream: ALWAYS start with a version at the start of that binary format! I'm using this in the current project and I can modify the saving/loading code for level data and still be able to load an older level to save it in the new format.

[mrt_]

you bringing up that it's a hexadecimal representation of a binary reminds me of the bell curve memes.
(beginner) It's a binary file
(middle) It's hexadecimal
(expert) It's a binary file

6:47 I don't have to imagine. I had to edit an Eye of the Beholder II (90s game) savefile to cheat on the game. It was something like "save the game and write down the number of <whatever you wanted> you have. Now throw one and save again. Compare both files to find the hex values and there is the byte you have to change... maybe".

[WladaK]

For me one more good reason for text is future changes in the data format. If you add/remove some variables in your format text parsers will easily handle old versions by using defaults/ignoring some keys. But binary ones will need added logic for each change and there needs to be version of the format saved in the data. Of course you should always have format version in all data even the text ones.

[SomeRandomPiggo]

I've only ever done binary serialization in C/C++, in Go it is super easy to serialize to text (JSON) so I use that in most of my Go projects. Interesting video!

[KojoBailey]

13:27 Absolutely! That's the kind of stuff I work with atm with modding, and optimisation is always something I'm seeking to learn

[zoltanujszaszi]

Nice comparison, I would love to see the code as well in a future video!

[JoelJose]

Can you do json vs yaml? Which is better for storing data, readability, commenting, which is better to use as config file? Are there better formats ?

[KoraktorXV]

1:35 no there is no link in the discription

[Voidload]

Nice vid! Always has thoughts on this topic - Unity serializes in YAML text file, UE serializes in binary.

Personally I think the most ideal engine would store assets in text format in editor and during cook process would generate a binary version for that build for smaller size and some "encryption" of data, merging blueprints sucks merging scenes in Unity is somewhat doable from my experience

[andresilvasophisma]

Another reason for using text format is if your software runs on multiple hardware configurations/operating systems.
In web development it's common practice to use json serialization for almost everything.

[custard]

while it still comes down to whether the format is intended to be read by a human or not, another reason/side effect for using text formats in the shipped build is modding

on i can still remember is in GTA San Andreas the data for vehicle specifications was in a text file (csv iirc) and allowed for some fun things like insanely fast tank

while determined users will still find a way to parse and + attempt to edit anything the text serialization does make it easier which depending on whether the game devs want to encourage/discourage may be another factor one way or the other

[Akronymus_]

Personally, for hexeditors, I prefer imhex, because it provides a sort of scripting interface to automatically annotate data in the files.

[ZulnorineP_Coding]

My best programming and my favorite teacher in my whole Life.Thanks to you Sir very Thanks....

[mdhvg]

I was just looking for this. U really read my mind.
Thank you so much

[ITR]

In the game I'm currently working on we store quests as json in the editor, but use MemoryPack for the runtime

[andrasbradacs]

Binary files contain data of a fixed length, and text files are also binary, with a lot of line breaks and a lot of human readable numerical values, which must be converted back from ASCII to numerical values ​​that can simply be placed in memory.