Jump to content

matzman666

Members
  • Posts

    2
  • Joined

  • Last visited

Posts posted by matzman666

  1. On 1/2/2019 at 5:04 AM, gprovost said:

    @GeckoX Good to hear you figure it out. Yes OMV is the easiest approach to setup your NAS without the need to be too much Linux savvy.

     

    @Koen No there is no microSD card included in the kit.

    Some users of the 1st batch got a free microSD card but it was a free goodie from the PCBA factory to apology from their repeated delay.

     

    OMV may be the easiest approach, but keep in mind that your performance will significantly suffer when using OMV.

    1. OMV only supports raid creation using the whole disk, which leads to a significant performance loss according to my experience.
    2. When using the OMV encryption plugin to create your dmcrypt device, an encryption algorithm will be used that is not supported by the hardware encryption engine, which again leads to suboptimal performance. There is no way to change the encryption algorithm in OMV.
  2. I also want to share my numbers because I think they show some interesting findings.

     

    My setup: Helios4 & Raid5 with 4x Seagate Ironwolf 4TB (ST4000VN008)

    To ensure comparability all numbers where obtained using the same command as used by @djurny: dd if=/dev/zero of=/mnt/removeme bs=1048576 count=16384 conv=fsync

     

     

    First of all some numbers which show how important correct alignment is (no encryption yet, filesystem was btrfs):

    • md-raid5 using the whole disk created via OMV or the instructions found in the wiki here:
      • Write-Throughput: ~76 MB/s
    • md-raid5 using optimally aligned partitions (created via parted -a optimal /dev/sdX mkpart primary 0% 100%):
      • Write-Throughput: ~104 MB/s

    That is a difference of about ~26%! Based on this numbers I would not recommend using the whole disk when creating a md-raid5. Using partitions is not supported at all by OMV, so the raid has to be created on the command-line.

     

     

    Now my numbers when using encryption:

    • md5-raid & luks (using aes-cbc-essiv:sha256) & xfs:
      • Write-Throughput: ~72 MB/s
    • md5-raid & luks (using aes-cbc-essiv:sha256) & btrfs:
      • Write-Throughput: ~66 MB/s
    • md5-raid & luks (using aes-cbc-essiv:sha256 with marvell_cesa kernel module unloaded) & btrfs:
      • Write-Throughput: ~34 MB/s
    • md5-raid & luks2 (using aes-cbc-essiv:sha256) & btrfs:
      • Write-Throughput: ~ 73 MB/s

    Looking at the numbers I see a performance loss of about 30% when using encryption. Hardware encryption is working and definitely speeds up encryption because when I unload the kernel module I see a performance loss of about 60% compared to the unencrypted case.

     

    When creating the luks partition via the OMV encryption plugin aes-xts is used by default which is not supported by marvell_cesa, and there is no way to configure a different encryption algorithm on the web-gui. To be able to use aes-cbc the luks partition has to be created via the commandline.

     

    Using luks2 instead of luks gives a bit of a performance boost. Luks2 is only supported by the Ubuntu image, the Debian image has the usual Debian problem: too old packages.

     

    Based on these numbers I am ditching Debian and OMV and are moving to Ubuntu. OMV is of little use, because for best performance I have to setup everything via the commandline. Also luks2 is more future-proof and results in better performance.

     

    Edit2:

     

    I played a bit more with luks2, and found something very interesting: With luks2 you can also change the sector size.

     

    The default sector size is 512 Byte, but when I change it to 4K, then I see massive performance improvements:

    • md5-raid & luks2 (using aes-cbc-essiv:sha256 and 4K sectors) & btrfs:
      • Write-Throughput: ~ 99 MB/s

     

    That's only a performance loss of ~5% compared to the unencrypted case!?! Before I had a performance loss of at least ~30%. I double-checked everything, and the numbers are real. This means luks2 is definitely the way to go.

     

    To sum up. To get the best performance out of an encrypted raid5, you need to:

    1. Install the Ubuntu image. This means you cannot use OMV, but that's the price to pay for best performance.
    2. Create a single partition on each of your disks that is optimally aligned.
      • parted /dev/sdX mklabel gpt
      • parted -a optimal /dev/sdX mkpart 0% 100%
    3. Create your raid with mdadm and pass the partitions you created in the second step.
      • mdadm --create /dev/md0 --level=5 --raid-devices=4 /dev/sda1 /dev/sdb1 /dev/sdc1 /dev/sdd1
    4. Create a luks2 partition using aes-cbc-essiv:sha256 and a sector size of 4K.
      • cryptsetup -s 256 -c aes-cbc-essiv:sha256 --sector-size 4096 --type luks2 luksFormat /dev/md0
    5. Create your file-system on top of your encrypted raid device.

     

     

     

    On 12/29/2018 at 7:15 AM, djurny said:

    Averaged throughput measurement:

    • Cipher: aes-xts-plain64:sha1, key size: 256bits
      • Throughput => 78.0 MB/s,
      • Interrupts for crypto => 11 + 8,
    • Cipher: aes-cbc-essiv:sha256, key size: 256bits
      • Throughput => 102 MB/s,
      • Interrupts for crypto => 139421 + 144352,
    • Cipher: aes-cbc-essiv:sha256, key size: 128bits
      • Throughput => 103 MB/s
      • Interrupts for crypto => 142679 + 152079.
     

    Do you have any numbers for the unencrypted case? I am curious because I want to know if you also see a performance loss of about 30%.

     

    Edit:

    Should be parted -a not parted -o.

×
×
  • Create New...

Important Information

Terms of Use - Privacy Policy - Guidelines