I'm not an expert in this area whatsoever, but this problem looks like an upstream kernel issue, and perhaps is related to this fallback deadlock fix? https://git.kernel.org/pub/scm/linux/kernel/git/herbert/cryptodev-2.6.git/commit/?id=a2715fbdc6fc387e85211df917a4778761ec693d
The posted fix patch is only applicable to kernel v5.10; just in case you want to give it a try, I've uploaded a ToT test build of v5.10-rc3 with this kernel patch applied: https://drive.google.com/file/d/1Eg_cn3bEYsRFZ3vrP9Mc0fxJjPREczcW/view. Again, I'm not sure if this will fix the problem, but there are a number of changes in cryptodev here compared to kernel v5.9, so it might be worth a try.
5kft's post in Orange Pi R1 IPSec freezing Problem with large amounts of traffic / e.g. iperf3 was marked as the answer