filesystem for raid5

[EduardCH]

I've got my helios64 recently, and I wonder what type of FS to choose for main storage. I have five 4TB disk drives. I've installed OMV, made raid5 and format it with XFS. I use XFS because I work with this FS on Centos on my x86 servers, I don't use ext4 because I don't like when sometimes it starts to check filesystem during boot process. My setup is working perfect for a couple of weeks. I don't have any reboots or freezes or any issues that I've read on this forum. Now I think about migrating from XFS to BTRFS or ZFS. The main reason is to have file check-summing and scrub. Does anybody have btrfs with raid5 or ZFS with raidz1 on Helios64? Do you have any problems with it? Also, as I know, Synology uses btrfs on top of mdraid. Is it ok to use it like this on Helios64? I've read debates about this filesystems on this forum, but I would like to know if anybody is really using it on Helios64.

 

This is my version: Linux helios64 5.9.11-rockchip64 #20.11.1 SMP PREEMPT Fri Nov 27 21:59:08 CET 2020 aarch64 GNU/Linux

[Jer]

Hello, btrfs raid5 is not ready for production: https://btrfs.wiki.kernel.org/index.php/RAID56

By the way you could use raid1, raid1c3 or raid1c4 (2 3 or 4 duplicates, spread over all your disks)

[TRS-80]

7 hours ago, EduardCH said:

btrfs on top of mdraid

 

I am more familiar with ZFS than btrfs, however my understanding is that stuff like this is generally not considered a good idea in either case?

 

7 hours ago, EduardCH said:

I've read debates about this filesystems on this forum, but I would like to know if anybody is really using it on Helios64.

 

I guess you might be referring to the thread I started (and then tkaiser replied) about ZFS vs btrfs.  If not, I would give that a read.  It's a bit long but (IMO) pretty good debate between knowledgeable people (especially tkaiser, whereas I am low to medium level wizard at best ).  Even though I am one of participants, I like to believe I remain fairly objective (also I have done quite a bit of research since years) and therefore I personally would put a lot more stock in such debate than random person N=1 anecdotal experience who just so happened to have purchased a COTS NAS device.  This is not a slight towards Kobol customers by any means, many of whom are very knowledgeable; rather my point being simply owning a device and using a certain filesystem (without any other research) does not make anyone an authority on the subject, necessarily.

 

So, getting back to your question.  From what I have read in this (Kobol) forum, I think that (quite?) a number of people are already using both btrfs and ZFS on Helios64.  Is that enough to convince me to start using it (on ARM)?  Personally, no.  But I am pretty conservative (paranoid? ) person by nature in general.  So, maybe I worry too much.    To give another data point, I can say that I spend quite a lot of time reading Armbian forums in general (not only Kobol Club) and I have yet to hear any stories about data loss or anything like that, with either of these advanced filesystems[0].  So, maybe (probably?) you should be fine?

 

[0] Caveat here being, it's still relatively early days (on ARM) in my view.  Of course, there was Helios4 before, and other devices, too, but many people just now receiving Helios64 so there has been a big uptick lately.

Edited December 9, 2020 by TRS-80
clarify

[gprovost]

Sound a bit strange to put ZFS on top of RAID5, since ZFS has raid feature : RAIDZ

Using ZFS on ARM is still a bit early days and requires a bit of tuning, therefore you need to understand ZFS a bit better than just on surface. For that you need to allow yourself some time to play around with it.

 

If you are looking at checksuming / scrubbing feature just on small portion of your data then there is also Snapraid tool that can be interesting.

 

It always comes don't to your usage. If you data is mainly non-encrypted data such as video, movie, picture then a bit flip | bit rotting is unlikely to have any effect.

 

It's hard to give a single recommendation that applies to everyone needs unfortunately.

[EduardCH]

2 hours ago, gprovost said:

Sound a bit strange to put ZFS on top of RAID5, since ZFS has raid feature : RAIDZ

Thanks for reply. I was talking about raid5 or raidz1. The idea is to have only one drive for parity.

 

2 hours ago, gprovost said:

Using ZFS on ARM is still a bit early days and requires a bit of tuning, therefore you need to understand ZFS a bit better than just on surface. For that you need to allow yourself some time to play around with it.

I already figured it out. I do have experience in ZFS on x86. But I decided for myself that I can not use ZFS (or btrfs) on ARM and feel safe for data at the same time. Will wait and see. I'm sure that this will change soon.

 

2 hours ago, gprovost said:

If you are looking at checksuming / scrubbing feature just on small portion of your data then there is also Snapraid tool that can be interesting.

This is exactly where I came to. I've set up SnapRaid and mergerfs. For home usage it's almost perfect.

 

2 hours ago, gprovost said:

It's hard to give a single recommendation that applies to everyone needs unfortunately.

You right, especially when there are not so much options, since ZFS and Btrfs are not ready yet. But I hope since we now have good ARM based hardware storage solution, like Helios64, this will change soon.