UART boot Issue Armada 3700 family with DDR4

[Thomas95]

Hello Everyone!

In the company where I am, we are migrating the current Zynq architecture with DDR3 to Armada 3700 family with DDR4.

Before deciding to migrate, we bought some espressobin and were delighted with this product.

We have therefore attempted to develop two cards that use the armada A3700 processor without success. We concluded that the problem is HW but we don't know exactly where.

 

I have done numerous tests. I try to summarize the key points as briefly as possible.

Our board uses DDR4 MT40A512M16LY-062E IT:E with Armada 88F3720-A0-BVB2I080-P123.

We have created a TIM, WTMI, Uboot package using A3700-utils-marvell, mv-ddr-marvell and atf-marvell from github as indicated in http://espressobin.net/espressobin-ultra-build-instruction/.

The package works correctly on espressobin V7. To test it we set the serial boot with the jumpers on the board then we used the tool A3700-utils-marvell/wtptp/linux/WtpDownload_linux .

We tried to throw the same thing on our board but uboot didn't start.

 

We then investigated further: to eliminate all the SW section above WTMI we modified the source code at A3700-utils-marvell/wtmi/sys_init/main.c by inserting a return 0 immediately inside the main (therefore after “u32 status”). On espressobin we see that the DDR switches correctly at 800MHz and the console goes back to the bootROM ones (therefore with the ">" symbol). On our board, however, the symbol is always that of the bootROM (we have never seen anything different) but the DDR does not switch at 800MHz: it remains at 400MHz instead.

 

Deepening the topic, several registers have been read from the bootROM console. In particular:

the section starting from address 1FFF_0000 onwards (at least 100 registers),

the section starting from 2000_6000 (at least 100 registers) and

the section starting at 6410_0000 have been read.

Sections 1FFF_0000 and 2000_6000 have consistent content between our board and espressobin (this is also consistent with TIM and WTMI file). The section 6410_0000 instead is consistent on espressobin with the uboot binary but it is not so on our board. Random and non-repeatable data are read following a power down of our board.

 

We have therefore concluded (I kindly ask for your confirmation) that the problem is the DDR. To rule out any possibility, our DDR was mounted on a V7 espressobin. The system starts correctly and has therefore validated the HW component.

 

In order to generate the package that contains the TIM (which from what we understand describes among other things some registers to allow switching to 800MHz) we use the following command:

export CROSS_COMPILE = / opt / toolchains / gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu / bin / aarch64-linux-gnu-

export BL33 = $ BINARIES_DIR / u-boot.bin

make DEBUG = $ MY_DBG USE_COHERENT_MEM = 0 LOG_LEVEL = 60 SECURE = 0 CLOCKSPRESET = CPU_600_DDR_600 DDR_TOPOLOGY = 5 BOOTDEV = SPINOR PARTNUM = 0 PLAT = a3700 all fip

CLOCKSPRESET = CPU_800_DDR_800 was also tried. The variations of this parameter are seen on espressobin (with oscilloscope on DDR clock) but not on our board (which remains at 400MHz).

 

Further analyzing the A3700-utils-marvell/tim/ddr/DDR_TOPOLOGY_5.txt file and the "ddr_tool" sources present in mv-ddr-marvell/. The ddr_tool are compiled by us with the following:

export PLATFORM = a3700

export DDR_TYPE = DDR4

make clean

make

We tried to change the value of CL and CWL. Putting wrong values also the espressobin stops (like our board) but we were not able to start our board instead.

 

From what we understand (I kindly ask for your confirmation), the problem is that the TIM structure does not correctly describe our DDR4 (or our PCB with the correct timings on the tracks) this causes the check on the checksum of the uboot section made by bootROM to fail. Then reset the chip.

 

We therefore ask for confirmation on our deduction and if there is any tool capable of giving us the correct values to be entered in ddr_tool to correctly set the communication with DDR4 on our board.

 

At the HW level we checked the DDR4 signals (with eye diagrams too). We do not see important reflections on signals that could compromise correct communication. However, the DDR4 clock always remains stationary at 400MHz and the data contained in it always seems to be random (read with command r 6410_0000 and following from the bootROM console).

Hope it can help you with problem analysis,

Best regards

[Pali]

DDR4 is really problematic on A3720. More people here on the forum have issues with Espressobin v7 which has 1.2 GHz CPU and DDR4 memory. There are more undocumented HW issues with DDR4 which are not mentioned in errata documents. You can find something mentioned in commit messages of public marvell git repositories. Last time when I worked with DDR4 on A3720 I saw that DDR training was extremly slow.

 

Anyway when building A3720 firmware code, you should use upstream instruction and not some from espressobin website which use ten years old code.

See: https://trustedfirmware-a.readthedocs.io/en/latest/plat/marvell/armada/build.html

 

But the best what you can do with DDR4 is to contact your Marvell FAE.

 

Or use DDR3, I had less problems with DDR3 on A3720.

[Thomas95]

Thanks for the support Mr Pali. We performed several tests including soldering our Armada 3720 and DDR4 on the V7 Espressobin and we didn't find any anomalies during the boot phase through the UART.

The use of the atf-ntim.txt file on our layout, on the other hand, has problems on the third boot phase of the OBMI file.

 

Consequentially, we have some questions about the excel file "DDR3_4_Static_configuration_tool_v1.1":

• Is it possible to have a description of the confidential registers contained in the document?
• Do these have any relation to the physical layout of the tracks on the PCB? Do they possibly need to calibrate the DDR4 signals according to the delay generated by the printout?
• How can we training DDR4? Can we use "write leveling"?

 

[Pali]

5 hours ago, Thomas95 said:

Consequentially, we have some questions about the excel file "DDR3_4_Static_configuration_tool_v1.1"

 

I do not know that excel file. Anyway all those questions are really HW specific and you should discuss it with Marvell FAE or other Marvell contact / supplier. Basically Marvell FAE is who has access to the confidential registers and who can share it to Marvell customers.

 

DDR training is done in software - firmware which you transfer over UART (in your case). I have not seen any (useful) documantation about DDR training for A3720. Source code is open source (but it is for me just a magic, I do not understand it) and available at: https://github.com/MarvellEmbeddedProcessors/A3700-utils-marvell/tree/master/wtmi/sys_init/ddr

 

Before DDR training (and even before transfering/loading CM3 firmware) is called TIM bytecode, part of which is static DDR configuration. This bytecode is simple write values into registers and configured DDR for static low speed mode. This code is generated during building from DDR_TOPOLOGY.

 

So for debugging DDR, it is needed to check if DDR is working fine after executing TIM bytecode (just is very slow) and it is DDR training who break RAM access. Or if even static low speed configuration of DDR via TIM bytecode makes RAM access not possible.