Choosing the Right Filesystem for Your RAID Setup

You have lots of stuff, photos on your phone, bookmarks for your favorite sites, movies, music, memes...lots of stuff..Where do you store it? If you want to backup your phone, laptoptop, desktop where do you put it? So you think you want a Network Attached Storage (NAS), or some kind of storage system for all your stuff...how do you choose the right one? This isnt a product guide, this is part one of a do-it-your-self guide to building one yourself using off the shelf components. This first step picking the right filesystem for the the job.

THIS VIDEO IS TECHNICAL, its the planning part...Later videos will be more user friendly...

You could just go buy an off the shelf solution and many people do, but if you have ever worked around large companies much, why do they for the most part build it themselves? What do they know that you dont?

We arent going to answer all of those questions today, but this series is about the do-it-your-selfers who want the bes while saving a few bucks in the process. And best of all you know how it works and there is never a subscription fee.

So lets take the first steps finding the right filesystem for our Storage Devide, we call them RAIDs because they are Redudant, Arrays of Independent Disks.

This ist for everybody...like i said most just go buy something they hope will work for their needs, and most often not they dont live up to that, commercial devices strike a middle of the road approach, a one size fits all...and usually they are left shoes on a right foot...so lets begin

Today we are going to look at some testing I did to try and answer that question.

Chapters
00:00 - Intro
00:49 - What's up with ZFS?
02:10 - ZFS Fuse
03:27 - Hardware Used for Test
05:47 - Software and System Config
06:44 - Test Description
06:54 - How to build the RAIDs?
07:45 - Benchmark - Initial Write
10:14 - Read
12:15 - Re-Read
13:20 - FRead
14:24 - FWrite
16:35 - Mixed Workloads
17:41 - PRead
19:21 - PWrite
20:26 - Random Read
21:22 - Random Write
22:30 - Re-Writes
23:36 - Reverse Reads
24:29 - Stride Read
25:35 - GeoMean All Tests
27:39 - A Few Thoughts

Follow me:
Twitter @djware55

[jholloway]

The beard is just getting more and more glorious

[_sneer_]

Fantastic video! I would love to see JFS thrown into the mix. I know it is kind of ancient and it is a lot of work to test a filesystem, but it is the only filesystem I have never (in more than two decades) had a single problem with and comparing to EXT4 a disc has 10% more usable space on it after formatting. That means a free 1TB with every 10TB of a disc, which effectively brings the cost of disk space down. It is also very CPU efficient and from what I remember from old tests it was very fast in every task.

[bertnijhof]

If you ignore the caching, you ignore a major part of the design of bcachefs; btrfs and OpenZFS. Those file system are designed to optimize all resources not only disks, but also memory caching for those disks. I'm close to 80 and I get the feeling, that you don't really appreciate modern designs anymore.
For 6 years I'm a user of ZOL and OpenZFS and I love it, that on a desktop 99% of my actions are served from my 4 GiB lz4 compressed cache L1ARC :)
I always compress data, because advantages are: less disk space; more effective cache space use and higher throughput (caching and compression).

Another comment, realizing that OpenZFS stays behind with only 6.8 support, still for comparison use a system, that supports all filesystems. Use kernel 6.8. and if needed another distro. Most distros do not yet support kernel 6.10, like the distros based on Ubuntu, Debian or OpenSUSE Leap. Anyhow avoid old intermediate layers!

[SIackware]

Oh perfect timing, I got a HP Proliant microserver gen8 yesterday which I plan on using to learn about RAID setups. This is just the video I needed to see, thanks DJ Ware!

[colinstu]

TIL: Avoid FUSE. Thought XFS would perform better too. Surprised Btrfs running so well, it kinda stunk in your prev non-raid tests.

[tkenben]

What would you speculate the performance be of llvm zfs? Something similar to weird numbers outside the borders like bcachefs and btrfs?

[roryboyes]

I'm currently looking for a mirroring solution that allows me to remove a drive from the raid & mount it on another machine without any hassle.
I know, raid is not a backup, but nobody seems to speak about the fact no matter how much redundancy you build into the raid, the host system is still a single point of failure.

I was hoping mdadm + raid1 could provide this but I don't think that's the case.
Any advice?

[johnsmith]

Could you clarify how you configured bcachefs and zfs? Neither of them have defined raid5/raid10 profiles, as far as I am aware

[ridcully]

Why not compile the latest 2.2 rls of zfs on that fedora?

[Felix-vehs]

I though Mebibytes is just base 2 (1024 KiB) instead of Megabytes which is base 10 (1000 KB) ?

[carpetbomberz]

Q: Bcachefs/BTRFS "inflationary" results, are they gaming the benchmarks somehow? How are they going faster than the actual bandwidth?

[savagepro]

DJWare: "Choosing the Right Filesystem for Your RAID Setup"
All Cyber Roaches: "RAID? FCUK! We're Dead🪳🪳🪳🪳"