matzman666

I also want to share my numbers because I think they show some interesting findings.
 
My setup: Helios4 & Raid5 with 4x Seagate Ironwolf 4TB (ST4000VN008)
To ensure comparability all numbers where obtained using the same command as used by @djurny: dd if=/dev/zero of=/mnt/removeme bs=1048576 count=16384 conv=fsync
 
 
First of all some numbers which show how important correct alignment is (no encryption yet, filesystem was btrfs):
md-raid5 using the whole disk created via OMV or the instructions found in the wiki here: Write-Throughput: ~76 MB/s md-raid5 using optimally aligned partitions (created via parted -a optimal /dev/sdX mkpart primary 0% 100%): Write-Throughput: ~104 MB/s That is a difference of about ~26%! Based on this numbers I would not recommend using the whole disk when creating a md-raid5. Using partitions is not supported at all by OMV, so the raid has to be created on the command-line.
 
 
Now my numbers when using encryption:
md5-raid & luks (using aes-cbc-essiv:sha256) & xfs: Write-Throughput: ~72 MB/s md5-raid & luks (using aes-cbc-essiv:sha256) & btrfs: Write-Throughput: ~66 MB/s md5-raid & luks (using aes-cbc-essiv:sha256 with marvell_cesa kernel module unloaded) & btrfs: Write-Throughput: ~34 MB/s md5-raid & luks2 (using aes-cbc-essiv:sha256) & btrfs: Write-Throughput: ~ 73 MB/s Looking at the numbers I see a performance loss of about 30% when using encryption. Hardware encryption is working and definitely speeds up encryption because when I unload the kernel module I see a performance loss of about 60% compared to the unencrypted case.
 
When creating the luks partition via the OMV encryption plugin aes-xts is used by default which is not supported by marvell_cesa, and there is no way to configure a different encryption algorithm on the web-gui. To be able to use aes-cbc the luks partition has to be created via the commandline.
 
Using luks2 instead of luks gives a bit of a performance boost. Luks2 is only supported by the Ubuntu image, the Debian image has the usual Debian problem: too old packages.
 
Based on these numbers I am ditching Debian and OMV and are moving to Ubuntu. OMV is of little use, because for best performance I have to setup everything via the commandline. Also luks2 is more future-proof and results in better performance.
 
Edit2:
 
I played a bit more with luks2, and found something very interesting: With luks2 you can also change the sector size.
 
The default sector size is 512 Byte, but when I change it to 4K, then I see massive performance improvements:
md5-raid & luks2 (using aes-cbc-essiv:sha256 and 4K sectors) & btrfs: Write-Throughput: ~ 99 MB/s  
That's only a performance loss of ~5% compared to the unencrypted case!?! Before I had a performance loss of at least ~30%. I double-checked everything, and the numbers are real. This means luks2 is definitely the way to go.
 
To sum up. To get the best performance out of an encrypted raid5, you need to:
Install the Ubuntu image. This means you cannot use OMV, but that's the price to pay for best performance. Create a single partition on each of your disks that is optimally aligned. parted /dev/sdX mklabel gpt parted -a optimal /dev/sdX mkpart 0% 100% Create your raid with mdadm and pass the partitions you created in the second step. mdadm --create /dev/md0 --level=5 --raid-devices=4 /dev/sda1 /dev/sdb1 /dev/sdc1 /dev/sdd1 Create a luks2 partition using aes-cbc-essiv:sha256 and a sector size of 4K. cryptsetup -s 256 -c aes-cbc-essiv:sha256 --sector-size 4096 --type luks2 luksFormat /dev/md0 Create your file-system on top of your encrypted raid device.  
 
 
Do you have any numbers for the unencrypted case? I am curious because I want to know if you also see a performance loss of about 30%.
 
Edit:
Should be parted -a not parted -o.