I found that this is related to the DRAM parameters in u-boot. I modified these parameters and now it can boot on my RADXA Cubie A5E v1.2 with 1GB RAM. However, I am not sure if these parameters are valid for the 2GB/4GB version that uses 0.6V.
This is the image with the parameters I modified: radxa-cubie-a5e-armbian-build
This is the tool I use to obtain parameters: radxa-cubie-a5e-u-boot