I had to apply the patch from pask on top of the emmc after using "nand-sata-install" to get it to boot, even after already applying it to the SD, I can only assume the bootloader it copies to the emmc is the unpatched variant.
After doing so it happily boots from emmc using nvme as root.
Using the minimal buster (4.4) build for the M4 with pask's patch everything seems to work as it should (wifi etc.)