I have had usb3.0 problems with my NanoPC-T4, close relative of your NanoPi-M4 but not really the same.
When using my ZWO ASI294MC pro camera (USB3.0) the first image in any sequence is saved, but then the software goes into a cycle where the countdown continues for the next image, reaches zero but the image is not saved, this continues over and over. To reset, I need to shutdown the software and re-boot the NanoPC-T4. The camera works fine when plugged into one of the NanoPC-T4 usb2.0 ports, it just saves images at the usb2.0 speed. That isn't a big problem for me as I just do DSO imaging anyway. Still it would be nice to have it working as I need all three usb ports.
There are reports from NanoPi-M4 users on the indilib.org forum having the same problem as I have with the ASI294 camera and their usb3.0 ports.
A few weeks ago I burned the Armbian 5.98 image to an SD card, then upgraded the kernel to 5.3 from Armbian-Config and the usb3.0 problem with my camera was cured, however networking did not work so I am still using the 4.4.192 kernel for now.
If you have an extra SD card you could give the 5.3 kernel a try.
Sorry I can't be more helpful