I have the same issue and i "solved" it using a usb3 hub between the drive and the rock64, its more like a workaround than a fix but maybe it can help you too, in my case said hub can be powered but its not relevant since the issue goes away no matter if i power it or not...and its a noname hub, if you need any extra information tell me how...also i dont have a serial interface yet, gonna get one in a week or two when i have some extra time...
I commented about this on the same post you reference on the ayufan github...