My topic (my February 9 post) was merged into this one but it turns it incorrectly. Now that I've debugged it, and can see my original problem more clearly, it was basically this: the problem I had in was that the idbloader.img (loaded from the eMMC) tries to boot the next stage (u-boot.itb) from the SD card, because of the patch in patch/u-boot/u-boot-rockchip64-dev/board_rockpi-4b/001-boot-order-prioritize-sd.patch and instead I wanted to boot all stages from the eMMC and use the SD card as storage. But if the SD card is present, armbian's idbloader tries to load later stages from there and then crashes. I'll attach what I've seen on the serial console, just for reference if someone has the same probem.
The solution was to simply delete the 001-boot-order-prioritize-sd.patch file and build the image like that. In that case, the whole boot sequence is loaded from the eMMC and I can use the SD card as storage.