@c0rnelius graciously submitted GitHub PR to upgrade u-boot that effectively removed the patch in question from the equation for the Radxa Zero.
While it was nice to "boot USB first", since it enabled a few interesting scenarios (boot non-Armbian EFI OSes in USB disk), it's not worth having it if normal Armbian operation is impacted.
I've advocated for this usb-first style to be default around u-boot 2022, but we're now moving to board-specific support (and removing completely in the case of the Zero) for 2023+.