In-depth: ELF - The Extensible & Linkable Format

ELF files are everywhere: From your PS4, to the Wii, to any Linux device out there. In this video we explore the Extensible & Linkable Format, and understand how it works - on the byte level!

Errata:
- At the beginning I say extensible while displaying executable and linkable file format - Extensible is the old name, nowadays ist called Executable. Must’ve gotten confused :)

Further reading:

Timestamps:
0:00 Intro
0:47 segment and section differentiation
2:05 ELF structure
3:39 ELF header data structure
9:34 Program header data structure
13:54 Section header data structure

[MatteoBucci]

Incredible how a format developed so much time ago is still capable of being fully functional without any need of a version change. That tells a lot on the foresight of the creators!

[connorhorman]

The difference between the program headers on 64-bit is for layout. The fields in the elf headers are aligned to their natural alignment, so 8-byte types are likely going to want to be aligned to 64-bit. It was done to move the 32-bit members together so the header wouldn't have to contain unnecessary padding.

[karlxu]

0:47 segment and section differentiation
2:05 ELF structure
3:39 ELF header data structure
9:34 Program header data structure
13:54 Section header data structure

[aurelienlevra]

Most pedagogic content about a presentation of how the ELF format work.
Top tier quality content

[bonbonpony]

13:00 Ah, the infamous `p_align` field that *everyone understands wrong* :P This is _not_ about 4-byte vs. 8-byte alignment (and you would know that if you ever tried hand-crafting ELF files or fiddling with existing ones), but it is meant to facilitate *paging* . Many people say that segments have to be aligned to page boundaries, but *this is also wrong!* The only true requirement is that their _offsets_ in file and in memory must be *congruent modulo the `p_align` value* (which is usually some reasonable multiple of the page size). The reason for this is that when segments from the file are memory-mapped (using `mmap`), the granularity of that mapping is the page size (i.e. you can only map whole pages), but the address is usually assigned by the kernel, and it might change on different runs. This isn't a problem, though, as long as data and instructions keep the same offsets from the page boundary. Then they can be easily mapped to different addresses in different address spaces, and yet their offsets from page boundaries stay the same as they were in the segment in the file. If the offset in the file were different than in memory, the system loader would have to shift it so that the offsets matched again, but then the beginning of the segment in memory would have to cross the page boundary, so the loader would have to map an additional page in front of our page, with lots of padding at the beginning. It would impact performance, memory usage, and be much more complicated to implement. Hence the alignment criterion that makes sure that when segments are mapped into memory, the offsets in the file and in memory match each other (i.e. they're the same modulo `p_align`).
13:16 This is because of machine word alignment. Two `Word`s are 64-bit together, so the next field will be aligned to 64-bit machine word boundary. Otherwise a padding would be needed which would only waste space. But since this padding is precisely the size of `p_flags`, they decided to rather move the `p_flags` field in there than waste this space for paddings.

[soveu]

i just started writing a bootloader and this is exactly what i need :)

[Solving_Engineer]

Wow this is the best elf video that i saw

[saschakaupp]

Awesome tutorial! Haven't looked into what happens inside an ELF since the time when Amigas used 68k and PowerPC simultaneously!

[LiEnby]

Oh, acturally ELF is *not exactly* the executable format of the PS4,

Rather 'SELF' which is like ELF but with some extra stuff added at the start mostly some boot flags, authentication & permissions info if its unencrypted (only on devkits) etc and then the actural elf header and stuff

Though it's often pretty easy to strip away the SELF stuff (and sometimes decrypt its contents too..) which basically just converts your SELF into a standard ELF binary- but the console cannot run ELF directly !

Oh also SELF is used on the PSVita and PS3 as well. and the format of SELF is different on all there consoles..

Lol on PSVita SELFs acturally have 2 ELF headers lol not sure about PS4 though.

Only PS2 and PS1 use ELF directly!

[valeyard]

Great explanation! I love digging into executable formats... wrote my own disassembler and have PE/COFF/ELF/etc loaders. My Elf reader code converts the ELF32 structures to ELF64.

[tolisec]

Very concise and precise overview of the ELF format. One of the best, if not best summary of the elf, program and section headers

[glitchy_weasel]

Really interesting video!
If anyone is interested in learning about the execution of programs in Linux, I recommend a series of articles by LWN titled 'How programs get run', and its follow up 'How programs get run: ELF binaries'. It explains how the OS loads executables into memory and how it invokes an interpreter on them if needed (eg. the shell for scripts, or the ELF program loader for dynamically-link programs).

[namesurname]

Hello to ITMO students, who are trying to write an elf parser for Risc-V. And thanks for an excellent video explanation.

[CyReVolt]

Watching this again for recap 2 years later now because the title image got me... Extensible was the old name, now it's Executable, and you even mixed both here. Wow 😮

[muskit_]

the editing style reminds me of retro games mechanics explained! nice explanations, though maybe a touch too fast-paced in explanations

[jwaffe]

Very well presented video, it would be interesting to see more details on the symbol table and how it compares to PE binaries and pdb files on Windows.

[qpnphq]

Thanks for a hugely informative description. I've been working with elf files for years without fully understanding the structure. Great job.

[clementpoon]

I'm developing my own operating system, and this is EXACTLY what i need.

[www]

This video was so well made and thought out! Kudos! :)

[joaofranciscobarata]

Awesome Work. I really enjoy these deeply technical videos, keep up with them.