Reverse Engineering - Computerphile

You just have the binary - can you work out what it does & how? Dr Steve Bagley talks about how you might reverse engineer a piece of software.


This video was filmed and edited by Sean Riley.

[tomquareme]

Recently Super Mario 64, Ocarina Of Time and even Jak & Daxter with its Lisp dialect GOAL got reverse engineered and even ported to pc. It's a work of art. For those interested: tools with disassemblers and decompilers like Ghidra or IDA are pretty helpful.

[ybergik]

Reverse engineering is a very useful skill to learn early on, but also one with a very steep learning curve. It happened to be how I learned programming back in the 80s on the C64 as access to any useful technical literature in my rural town was impossible to come by. I spent much more time in the machine code monitor examining games and demos to slowly learn how things were done than I ever did actually playing the games themselves. Perhaps it was a happy coincidence as I may not have gone that direction if I could simply have learned everything from reading books. Practising reverse engineering can greatly improve your analytical skills and pattern recognition as well as a more in depth knowledge of the underlying platform; skills that are, in fact, very useful in regular engineering. So learning reverse engineering enables you to become a much better engineer.

[eudorian]

I loved xoreaxeaxeax's talks about his movfuscator: First, he stumbles upon the fact that the MOV instruction turns out to be Turing complete. He then proceeds to write a compiler that turns any piece of code, even the compiler itself, into a program that ONLY uses MOV instructions. (At a hefty performance cost obviously.)
And he then spends his time making the reverse engineered flow graphs in IDA to render a picture of himself, and some profanities. It's at a next level.

[marsgal]

I maintain several legacy systems that were written in C. They were written in an idiosyncratic style, often by people who wanted to demonstrate to the world how smart they were. It can take a while to figure out just what the code is doing. Sometimes you get a head start: this code/data/whatever was written by the same people at about the same time, so it will resemble other things they've done.

[ecdhe]

One thing not mentioned in the video is that reverse engineering is more or less regulated depending on the country. For instance, my understanding is that, in the US, the person who reverse engineers some software and the person who implements a "clone" of said software need to be two different people. Furthermore, the latter must have no prior experience with the original software and must use only the specification written by the former person to write the implementation. The DMCA added further restrictions on reverse engineering. Other countries may have their own restrictions.

[Mutual_Information]

A very famous, difficult and consequential case of reserve engineering: Alan Turing and his team's cracking of the Enigma machine.

It's hard to think of more important technical problems to solve than that..

[danthest]

"Nothing is ever a waterfall in Computer Science except you are doing it wrong"
100% on point

[jeromethiel]

May not be completely true, but i watched a documentary about Compaq where they actually had a software engineer de-compile the IBM Bios, just to see how it worked. Once he did so, he reported to Compaq that "this isn't hard, there's nothing special here."

But since he had seen the code, he was not allowed to work on the project anymore, since he might be influenced by the code he had seen. Compaq had to hire an entire team of programmers to look at how the bios calls worked, without seeing the coded behind the calls, so that they could replicate the outcome, without necessarily replicating the IBM code that generated the original outcome.

So when IBM sued Compaq, they could legitimately claim that their compatible BIOS was not based on the code IBM had copyrighted from Microsoft. Nor was the Compaq DOS based on the code of Microsoft DOS.

[WunderWulfe]

Reverse engineering also applies to archives/files, i.e. watching for patterns in data and figuring out how it is used in order to reconstruct or convert/export. common practice for the modification or extension of a program’s assets. Most modern formats follow the structure of [header + chunk(s)], so finding the definitions leaves only making use of the data in each of the chunks

[Produkt_R]

I'm always amazed and overwhelmed by the sheer number of lines that even the smallest executable has when decompiled to assembly.

[carlospulpo]

Been Reverse-engineering for over 30 years, no University degree and learned everything from software design patterns to how the SoC bring up is done. You learn the real way it works by reverse-engineering.

I started out with C-64 and the super snapshot cartridge and moved on from there to the latest ARM hardware .

[ScottLahteine]

Some of my favorite (hardware) reverse-engineering going on in recent times is the work by CuriousMarc and his cohorts at the Computer History Museum, reverse-engineering the computers and radio equipment used in the Apollo missions. Their many-part series can be found here on YouTube.

[jeromethiel]

I have actually had to reverse engineer more than one industrial automation system with nothing more than a poorly drawn schematic (which wasn't up to date) and raw code.

First step was identifying all I/O points, so i knew what a particular input or output did. Then based on that you could figure out a lot of the basic logic of the machine. This is a start/stop circuit. This is a closed loop control system. This is a permissive or interlock logic block.

Still takes a lot of time, but it can be very rewarding when you finally get to the point where you can actually maintain and/or improve the machine automation.

[zacklayman]

Nothing's ever a waterfall unless u are doing it wrong. Beautiful

[TheInternetHelpdeskPlays]

The best example of clean room reverse engineering is GTA4.
One of the best trainer programs was built with the software in the next room. Engineers with gta looked at how certain actions worked in memory and passed their findings to totally separate engineers who worked out how to interrupt and alter what it did.

Rockstar tried to sue them for hacking and theft, but it failed because they hadn't built any software using any of the GTA code.

[KyuVulpes]

I am kind of surprised you didn't bring up the example of ReactOS being a reverse engineering of the Windows NT kernel. A truely massive undertaking since the NT kernel is big, complex, and probably bloated.

[trejkaz]

A whole video on what a space is would educate some people, that's for sure. It's practically impossible to get people to agree on it.

[dcc]

Season 1 of the TV Series "Halt and Catch Fire" (available on Netflix) shows the reverse-engineering technique explained early in this video. Although not specifically calling out Compaq, it's their story without the name :). Take it with a grain of salt, though. It shows them using a volt meter and oscilloscope to come up with the BIOS code <facepalm>...but the show is very entertaining overall.

[RealCadde]

You haven't mentioned reverse engineering to bypass copy protection/licensing.
I've reverse engineered a few small things that i liked but never had the opportunity to license.

In one case, it was as simple as finding the reference to a string that bothered you about purchasing a license and in the jump instruction simply make it so it never performs the jump and proceeds as normal.

In another case i reverse engineered the licensing key algorithm. Then i wrote a keygen for it.

Reverse engineering is a lot of fun, sometimes hair pulling frustrating, but a lot of fun and the moment you make a breakthrough you feel like you are on top of the world.

[lerssilarsson]

Already back in the 90s some GPU drivers recognized benchmarking and gave "incredible results". ;-)