Reduce Latency By 60% With ProtoBufs!!! | Prime Reacts

Recorded live on twitch, GET IN

MY MAIN YT CHANNEL: Has well edited engineering videos

Discord

[trapexit]

"I wrote bad software and made it less bad and now it is a lot better." News at 11.

[trapexit]

"Human readable" is a terrible requirement. It is trivial to build conversion tools.There are lots of binary formats that are easy to use and way faster and less memory hungry.

[dereckmelancon]

gRPC is great and the fact you define everything using IDL and can have the code generated for you is simply amazing.

[tytywuu]

Have been sneaking in protobuf for my project at work. Amazing!

[themohmand]

We use Protobuf in our embedded devices when they communicate over BLE to mobile devices. It's great and expanding the list of messages is super straight forward

[Telhias]

Personally I am partial to human typists and fax. Fax the required data to the station that needs it and have the typist input it manually. Bonus points for having every single fax sent to the archive afterwards in case it is needed at some point for say... debugging.

[salvatoreshiggerino]

The "human readable" thing seems like a relic from a really narrow window of time where people didn't think moving masks over octal dumps on tractor paper wasn't "human readable" enough, yet computers had too little spare memory to actually print out a structure on demand, which was maybe like one year back in the 1970s.

[williamclifford]

I have a lecture I deliver to undergrads on json, protobuff, thrift, and avro titled data encodings. It's one of my favourites to deliver each year and I can see the students really start to spring up and get interested as soon as I say json isn't their only option. Also the whole schema vs shemaless option and forward and backward compatibility are fun conversations to have. Makes for a fun coding lab too.

[srit]

gRPC is actually pretty good for communication between devices that are close to each other and don't require a central server, it's a way to decentralize the network and apply decentralized algorithms easily

[jaysistar]

gRPC does have something like Capnproto called flatbuffers, which has a different IDL. Using protobuf is not a requirement for gRPC.

[laughingvampire]

I have always been skeptic of micro-services because to me was just trying to imitate Erlang/Go from the UNIX side of things, making it so it can plug into the past UNIX ecosystem, and many Erlangers agree that it feels similar if you use k8s it feels similar to Erlang/OTP (orchestration framework for distributed programs in Erlang). But to me the problem is that a container is huge compared to an erlang process.

So to me the point is "why make a shitty copy when you can get the original"

[OneWingedShark]

ProtoBuf only exists because the developers didn't know about ASN.1; honestly, a bif of tooling (like a viewer) and ASN.1 would be a superb method for communication for a vast majority of this sort of stuff.

[pb]

We use protobufs for our internal server communication. And agreed it’s awesome

[nicwhites]

One huge benefit that I've used protobufs for which I haven't heard alot of people using is using protobufs for data models. Here me out: If you represent a majority of your data models in protobufs you can use them as a version controlled data model repo in which you can then cross compile them into other languages that you need ( that support protoc compilation). One of the issues I have ran into in a previous shop was having multiple language code bases that required data model syncing. It's not rocket surgery to keep them similar, but easy to forget and add time to projects. If you have a single repo that represents data models in a single language that can be cross compiled into other languages you don't have to worry about making sure your team is sync'd up across languages ( I'm looking at you json).

[alexpyattaev]

When protobuf will become able to do on-demand zero-copy deserialization, it will become actually cool.

[qmster]

There's no reason for human-readable when you can have self-describing. Realistically, for everyday objects shipping the whole schema together with the object is cheaper than shipping just the object as json.

[sharp]

"They must had JDSL before this..." cracked me up🤣

[daryl]

I reduced latency by 80% for JNI for a project awhile ago by serializing with protobuf. It was the best move ever.

[rdil]

JDSL defeated by Protobuf? Impossible. Tom is a genius.

[PellyFTW]

The performance increase shown in their own chart is roughly 10ms, the average LinkedIn user's load times would likely be more affected by the weather than this optimization. This is why percentages are misleading.