Helios4 - Cryptographic Engines And Security Accelerator (CESA) Benchmarking

[gprovost 21]

It's been a while I have on my TODO list : write a guide on how to activate and use the Marvell Cryptographic Engines And Security Accelerator (CESA) on Helios4.

 

Previously I already shared some numbers related to the CESA engine while using @tkaiser sbc-bench tool. I also shared some findings on the openssl support for the kernel modules (cryptodev and af_alg) that interact with the cesa engine. My conclusion was :

 

1. performance wise : effectively cryptodev performs slightly better than af_alg.

2. openssl / libssl support : very messy and broken, it all depends which version of openssl you use.

 

Since many Debian Stretch apps depend on "old" libssl (1.0.2), I felt taking the cryptodev approach was the best way since it could expose all encryption and authentication algorithms supported by the cesa engine... even though it requires some patching in openssl. Plus cryptodev implementation in new LTS openssl version 1.1.1 has been completely reworked, so long term it should be the right way.

 

Anyhow I'm not going to describe here the step by step setup, I'm already writing a page on our wiki for that, once it's ready I will post the link here. Also I won't risk myself talking about the relevance of some of ciphers, it deserves a topic on its own.

 

I'm just going to share benchmark number on a concrete use case which is HTTPS file download :

 

So I setup on my Helios4 Apache2 to serve a 1GB file hosted on a SSD drive.

 

Then I did 3 batch of download tests, for each batch I configured Apache2 to use a specific cipher that I know is supported by the cesa engine.

• AES_128_CBC_SHA
• AES_128_CBC_SHA256
• AES_256_CBC_SHA256

 

For each batch, I do the following 3 tests :

1. download without cryptodev module loaded (100% software encryption)

2. download with cryptodev loaded and libssl (openssl) compiled with -DHAVE_CRYPTODEV -DUSE_CRYPTODEV_DIGESTS

3. download with cryptodev loaded and libssl (openssl) compile only with -DHAVE_CRYPTODEV, which means hashing operation will still be done 100% by software.

 

Here are the results :

 

 

Note: CPU utilization is for both cores. Obviously each test is just a single process running on a single core therefore when you see CPU utilization around 50% (User% + Sys%) it means the core used for the test is fully loaded. Maybe i should have reported the number just for the core used, which would be more or less doing x2 of the value you see in the table.

 

For reference:

  Using AES_128_GCM_SHA256 (Default Apache 2.4 TLS cipher. GCM mode is not something that can be accelerated by CESA.)

  CPU Utilization : User %42.9, Sys %7.2
  Throughput : 30.6MB/s

  No HTTPS
  CPU Utilization : User %1.0, Sys %29.8
  Throughput : 112MB/s

 

 

CONCLUSION

1. Hashing operation are slower on the CESA engine than the CPU itself, therefore making HW encryption with hashing is less performant than 100% software encryption.

2. HW encryption without hashing provides 30 to 50% of throughput increase while decreasing the load on the CPU by 20 to 30%.

3. Still pondering if it's worth the effort to encourage people to do the move... but i think it's still cool improvement.

 

 

 

 

[gprovost 21]

I configured SSH to work with cryptodev and use cipher AES-CBC-128.

 

I did a scp download of the same 1GB file and get the following perf :

Throughput : 56.6MB/s

CPU Utilization : User %12.3, Sys %31.2

 

Pretty good :-)

 

Important note :

As concluded in previous post, in the case of Helios4, using cryptodev only for encryption and not for authentication (authentication involves hashing) is the only mode that provides some network performance and cpu load improvement. The other advantage of this mode, is that cryptodev will be completely skipped by sshd... because otherwise sshd will rise an exception during authentication because cryptodev try to do some ioctl() call that are forbidden by seccomp filter in sshd sandbox.

 

If you still want to test using cryptodev for ssh, the easy workaround is to use normal privilege separation in sshd instead of sandbox (UsePrivilegeSeparation yes). Then as for apache2, you will have to force to use a cipher that is supported by the CESA engine (e.g aes128-cbc)... and most probably you will also have to do the same on client side.

 

Disclaimer: The sshd tweaking is not recommended for security reason. Only experiment with it if you know what you are doing.

 

 

For reference with Cipher encryption algo not supported by CESA :

AES128-CTR

CPU Utilization : User %39.1, Sys %16.4

Throughput :  39.9MB/s

 

CHACHA20-POLY1305 (default cipher for ssh)

CPU Utilization : User %40.6, Sys %17.0

Throughput :  29.8MB/s

 

 

 

[gprovost 21]

Finally found the time to finish the CESA page on our Helios4 Wiki. It's not as exhaustive as it should be, but enough to help people experiment.

 

https://wiki.kobol.io/cesa/

 

For the ones who are interested, please have look... any comment welcome ;-)

 

[markbirss 2]

@gprovost thank you for effort with the guide.

 

Could you possibly include some CJDNS benchmarks ?

[gprovost 21]

On 11/20/2018 at 8:07 PM, markbirss said:

Could you possibly include some CJDNS benchmarks ?

 

I don't know much about CJDNS. Does CJDNS supports cryptodev or AF_ALG ?

[markbirss 2]

@gprovost  refer to these links for detail around how cjdns uses it's own CryptoAuth protocol with ed25519, poly1305, and salsa20.

https://github.com/cjdelisle/cjdns/blob/master/doc/Whitepaper.md

https://github.com/hyperboria/bugs/issues/112

 

To see if the encryption task the CJDNS uses benefits or not from cryptodev hw acceleration

 

[gprovost 21]

I already looked at the white paper but I don't have much time now to dig further. I'm not sure if CJDNS interfaces with the Kernel Crypto API.

 

Plus anyway the Marvell CESA engines don't support SALSA20 stream encryption... so i don't think CJDNS crypto can be accelerated.

 

 

[markbirss 2]

Ok, understood.

[Koen 0]

It could be interesting to see the test repeated while on a LUKS encrypted filesystem?

[gprovost 21]

@Koen Good point, will do a benchmark one of those days.