NAND reboot frequently fails on nanopi neo plus2, workaround


Recommended Posts

Armbianmonitor:

Dear all,

 

Starting from NAND memory used to work fine on the nanopi neo plus2, however with the new boards (both v1.2 and v2.0 revision) we just obtained it frequently doesn't want to reboot, while a cold boot works fine (I tried also earlier u-boot versions (2019.04, 2018.11) without much success for the reboot from NAND). The re-boot process stops at this point (from the serial console):

[  OK  ] Reached target Shutdown.                                                                             
[  152.452083] reboot: Restarting system  

U-Boot SPL 2019.10-armbian (Jan 25 2020 - 19:56:27 +0100)                                                     
DRAM: 1024 MiB                                                                                                
Trying to boot from MMC2

I suspected the problem to be in the u-boot, since the friendlyarm on NAND reboots without a hitch. I found a work-around for the latest stretch image (Armbian_20.02.0-rc1_Nanopineoplus2_stretch_current_5.4.14.7z), by doing the following:

  • boot friendlyarm image from NAND and copy the sunxi-spl.bin:
    • sudo dd if=/dev/mmcblk2 of=sunxi-spl-friendlyarm.bin count=4 bs=8k skip=1

  • boot armbian from NAND and update u-boot:
    • switched to linux-image-next-sunxi64=5.90 kernel via armbian-config / System / Other / switch

    • dd if=sunxi-spl-friendlyarm.bin of=/dev/mmcblk2 count=4 bs=8k seek=1 conv=fsync

The atached armbianmonitor -u upload is from after a fresh reboot with the working configuration.

 

Kind regards,

Dennis

Link to post
Share on other sites
Donate and support the project!

For your information, with this work-around, U-boot warns about a SPL version mismatch (DRAM:  sunxi SPL version mismatch: expected 3, got 2), but works nontheless. I wonder what is different in the SPL version that makes the boot possible (maybe the CPU frequency or DRAM frequency?), and if we can transfer this functionality to the newer SPL without breaking the reboot.

See the serial trace of a successful reboot below for the NanoPi neo plus hardware revision 2.0 (revision 1.2 with 512MB DRAM works as well):

[  311.266768] reboot: Restarting system                                                                      
                                                                                                              
U-Boot SPL 2017.11 (Nov 15 2019 - 05:14:40)                                                                   
DRAM: 1024 MiB(504MHz)                                                                                        
CPU Freq: 408MHz                                                                                              
memory test: 1                                                                                                
Pattern 55aa  Writing...Reading...OK                                                                          
Trying to boot from MMC2                                                                                      
Boot device: emmc                                                                                             
NOTICE:  BL31: v2.1(debug):bb2d778-dirty                                                                      
NOTICE:  BL31: Built : 18:34:11, Jul  5 2019                                                                  
NOTICE:  BL31: Detected Allwinner H5 SoC (1718)                                                               
NOTICE:  BL31: Found U-Boot DTB at 0x40899d8, model: FriendlyARM NanoPi NEO Plus 2                            
INFO:    ARM GICv2 driver initialized                                                                         
INFO:    Configuring SPC Controller                                                                           
NOTICE:  BL31: PMIC: Defaulting to PortL GPIO according to H5 reference design.                               
INFO:    BL31: Platform setup done                                                                            
INFO:    BL31: Initializing runtime services                                                                  
INFO:    BL31: cortex_a53: CPU workaround for 855873 was applied                                              
INFO:    BL31: Preparing for EL3 exit to normal world                                                         
INFO:    Entry point address = 0x4a000000                                                                     
INFO:    SPSR = 0x3c9                                                                                         
                                                                                                              
                                                                                                              
U-Boot 2019.04-armbian (Jul 06 2019 - 18:02:12 +0200) Allwinner Technology                                    
                                                                                                              
CPU:   Allwinner H5 (SUN50I)                                                                                  
Model: FriendlyARM NanoPi NEO Plus 2                                                                          
DRAM:  sunxi SPL version mismatch: expected 3, got 2                                                          
1 GiB                                                                                                         
MMC:   mmc@1c0f000: 0, mmc@1c11000: 1                    

An occasional successful boot from NAND without the workaround looks like below. The main difference seems to be the cpu/memory frequency and the memory test.

[  401.171972] reboot: Restarting system                                                                      
                                                                                                              
U-Boot SPL 2019.04-armbian (Jul 06 2019 - 18:02:12 +0200)                                                     
DRAM: 1024 MiB                                                                                                
Trying to boot from MMC2                                                                                      
                                                                                                              
U-Boot SPL 2019.04-armbian (Jul 06 2019 - 18:02:12 +0200)                                                     
DRAM: 1024 MiB                                                                                                
Trying to boot from MMC2                                                                                      
NOTICE:  BL31: v2.1(debug):bb2d778-dirty                                                                      
NOTICE:  BL31: Built : 18:34:11, Jul  5 2019                                                                  
NOTICE:  BL31: Detected Allwinner H5 SoC (1718)                                                               
NOTICE:  BL31: Found U-Boot DTB at 0x40899d8, model: FriendlyARM NanoPi NEO Plus 2                            
INFO:    ARM GICv2 driver initialized                                                                         
INFO:    Configuring SPC Controller                                                                           
NOTICE:  BL31: PMIC: Defaulting to PortL GPIO according to H5 reference design.                               
INFO:    BL31: Platform setup done                                                                            
INFO:    BL31: Initializing runtime services                                                                  
INFO:    BL31: cortex_a53: CPU workaround for 855873 was applied                                              
INFO:    BL31: Preparing for EL3 exit to normal world                                                         
INFO:    Entry point address = 0x4a000000                                                                     
INFO:    SPSR = 0x3c9                                                                                         
                                                                                                              
                                                                                                              
U-Boot 2019.04-armbian (Jul 06 2019 - 18:02:12 +0200) Allwinner Technology                                    
                                                                                                              
CPU:   Allwinner H5 (SUN50I)                                                                                  
Model: FriendlyARM NanoPi NEO Plus 2                                                                          
DRAM:  1 GiB                                                                                                  
MMC:   mmc@1c0f000: 0, mmc@1c11000: 1    

 

Link to post
Share on other sites

Thanks for this post Denny.

 

Armbianmonitor:

http://ix.io/2cPw

 

Can you or anyone actually, point me to a solution? I have a NEO 2 that will not reliably boot (now). Your solution looks like you boot SD card and then, using the booted system, fix the MMC boot process. I can't get board to boot with valid SD image. This board and image were working fine and I know of nothing I did that would break it. The same card/image works on a NEO 2+ just fine. Both are earlier HW revs I think but have been working reliably through many power cycles and shutdowns.  I see that no DTB is found now on my non booting board when it won't boot according to the console but unclear how to fix in this case. I don't want to break the SD card - I spent a lot of time getting it set up and was trying to backup card and ran afoul of this problem.

 

In my case I don't want to use mmc because I can't replace if I run into a wearout problem. I want to be able to drop in a new card with a reference backup image and have the board back operational. This may not be optimal but at my current level of proficiency and because of the application this seems the most sensible right now.  

 

So, to re-state more clearly the question,  what should I do to get a (more) reliable boot from sd card at this point? I am concerned that while the board is running at the moment, It will choke on some future reboot - perhaps in the process of fixing uboot and I won't know what to do... This isn't something I feel I can 'experiment' with to fix. There are too many unknowns for me.  Dumping a file onto the board or editing via winscp or webmin I can do now but once the board will not boot to SD I am in up a little over my nose... 

 

This board doesn't do much and doesn't need to do it particularly fast. Reliability is far more important in this case.

 

U-Boot SPL 2019.04-armbian (Jul 16 2019 - 09:15:50 +0200)
DRAM: 4096 MiB
Trying to boot from MMC1
NOTICE:  BL31: v2.1(debug):3ce3ce0-dirty
NOTICE:  BL31: Built : 17:00:24, Jul 15 2019
NOTICE:  BL31: Detected Allwinner H5 SoC (1718)
NOTICE:  BL31: No DTB found.
INFO:    ARM GICv2 driver initialized
INFO:    Configuring SPC Controller
NOTICE:  BL31: PMIC: Defaulting to PortL GPIO according to H5 reference design.
INFO:    BL31: Platform setup done
INFO:    BL31: Initializing runtime services
INFO:    BL31: cortex_a53: CPU workaround for 855873 was applied
INFO:    BL31: Preparing for EL3 exit to normal world
INFO:    Entry point address = 0x4a000000
INFO:    SPSR = 0x3c9

 

!!! This time the board booted as expected!!! Problem is I can't have it behaving like this at the remote site!

U-Boot SPL 2019.04-armbian (Jul 16 2019 - 09:15:50 +0200)
DRAM: 512 MiB
Trying to boot from MMC1
NOTICE:  BL31: v2.1(debug):3ce3ce0-dirty
NOTICE:  BL31: Built : 17:00:24, Jul 15 2019
NOTICE:  BL31: Detected Allwinner H5 SoC (1718)
NOTICE:  BL31: Found U-Boot DTB at 0x408c3b0, model: FriendlyARM NanoPi NEO 2
INFO:    ARM GICv2 driver initialized
INFO:    Configuring SPC Controller
NOTICE:  BL31: PMIC: Defaulting to PortL GPIO according to H5 reference design.
INFO:    BL31: Platform setup done
INFO:    BL31: Initializing runtime services
INFO:    BL31: cortex_a53: CPU workaround for 855873 was applied
INFO:    BL31: Preparing for EL3 exit to normal world
INFO:    Entry point address = 0x4a000000
INFO:    SPSR = 0x3c9


U-Boot 2019.04-armbian (Jul 16 2019 - 09:15:50 +0200) Allwinner Technology

CPU:   Allwinner H5 (SUN50I)
Model: FriendlyARM NanoPi NEO 2
DRAM:  512 MiB
MMC:   mmc@1c0f000: 0
Loading Environment from EXT4... ** File not found /boot/boot.env **

** Unable to read "/boot/boot.env" from mmc0:1 **
In:    serial
Out:   serial
Err:   serial
NanoPi NEO2 v1.0 detected
Net:   phy interface7
eth0: ethernet@1c30000
** Reading file would overwrite reserved memory **
There is no valid bmp file at the given address
starting USB...
USB0:   USB EHCI 1.00
USB1:   USB OHCI 1.0
USB2:   USB EHCI 1.00
USB3:   USB OHCI 1.0
scanning bus 0 for devices... 1 USB Device(s) found
scanning bus 1 for devices... 1 USB Device(s) found
scanning bus 2 for devices... 1 USB Device(s) found
scanning bus 3 for devices... 1 USB Device(s) found
       scanning usb for storage devices... 0 Storage Device(s) found
Autoboot in 1 seconds, press <Space> to stop
switch to partitions #0, OK
mmc0 is current device
Scanning mmc 0:1...
Found U-Boot script /boot/boot.scr
3042 bytes read in 2 ms (1.5 MiB/s)
## Executing script at 4fc00000
U-boot loaded from SD
Boot script loaded from mmc

Thanks to all.

 

Q

Link to post
Share on other sites
50 minutes ago, Quanta said:

So, to re-state more clearly the question,  what should I do to get a (more) reliable boot from sd card at this point? I am concerned that while the board is running at the moment, It will choke on some future reboot - perhaps in the process of fixing uboot and I won't know what to do... This isn't something I feel I can 'experiment' with to fix. There are too many unknowns for me.  Dumping a file onto the board or editing via winscp or webmin I can do now but once the board will not boot to SD I am in up a little over my nose...

 

Hi Quanta,

Thanks for you reaction. In my case the (re)boot from sdcard do not fail, I only had failing EMMC re-boots. I think the older first stage u-boot from FriendlyArm is running at slower speed so there is less chance of failures. Your u-boot on the sdcard may be fixable with a similar procedure, i.e. by capturing the first stage u-boot bytes from FriendlyArm NEO image and writing it to the sdcard instead of the MMC I guess it would be possible to do this on a linux PC without first booting from the sdcard. Before changing the sdcard, you can of course make a backup on your hard-drive: dd if=/dev/mmcblk0 of=recoveryimage.img bs=8k

 

Quote

In my case I don't want to use mmc because I can't replace if I run into a wearout problem. I want to be able to drop in a new card with a reference backup image and have the board back operational. This may not be optimal but at my current level of proficiency and because of the application this seems the most sensible right now.  

I think the flash wearing on regular sdcards would be similar to EMMC, most mobile phones run from EMMC nowadays and have sdcard as optional storage. Industrial-grade sdcards may be more reliable, but can still have degraded contacts and problems after vibrations. You can also start using sdcards once the EMMC starts failing, that's my plan with our remote Neo2+'s.

 

Kind regards,

Dennis

 

Edited by Dennboy
added notice about backing up the sdcard before starting
Link to post
Share on other sites
7 minutes ago, Onira said:

In my case the easiest way is just having empty SD card in slot and it reboots again but I will try Your solution, Dennboy

I guess the friendyarm image you mention about is the image from FriendlyARM wiki?

Yes, I use the image from FriendlyArm mentioned on their website (in my case, I booted the image that was on the MMC when it got shipped, which is most likely the same one). To capture their first stage bootloader with dd, I analysed the nand-sata-install script to see what needed to be copied. It may be possible to directly fetch the bootloader part from the sdcard image, with a similar dd with if=sdcard.img when you have a linux PC.

 

Since u-boot is opensource, we can probably ask FriendlyArm for their software/configuration changes to the first-stage u-boot in order to get this integrated into the Armbian tree as a patch.

Link to post
Share on other sites

Dennis I apologize for calling you Denny.  It wasn't intentional.  It was done in haste. I *always* check with the person for a preference before doing such a thing.

 

I am not clear just how to do this. What file has bytes? What form (binary? Ascii? ) How to ID bytes I want etc.

.....Deleted

 

I worked for a while trying to express more questions and observations, verify what I think I have seen etc. I wonder if it is possible I have stumbled on the source of the problem - usb to serial console adapter. Is it possible that it is messing with power? I am using the FA adapter sold with the boards. I use the FA power dock to power the board from a compact form factor PC supply 12V buss. In order to see boot, I leave serial adapter connected to console, powered and connected to Putty. I usually issue shutdown, toggle power switch on dock for some seconds and then re-power.  [I have observed in the past that after issuing shutdown the board does go through the routine but will get / stay warm -even hot long after if I don't shut of dock. Seems I read shutdown doesn't work like it ought to for early boards - possibly related? ] When board acts up I cannot do what I normally do in remote context (reboot now). I have a warning that comes up instead of shutdown when I issue that command without explicit path because normally I won't be there to power cycle.  Also, in the remote context, I never have the console port connected. So the greatest potential for this possible problem to appear would be when the board is on the bench and I have removed sd card to image, and then re-install while using console port to monitor.

...

 

A little more testing.

 

I have some trouble with the usb cable to the serial adapter dropping out if I move it so this complicates the testing some but it looks like this is part of the problem at least. There are times when with serial adapter connected, and the power dock switched off,  cpu power remains up. I see it on a protobard connected to the B side of the power dock (has a power LED). Removing usb extinguishes of course. Also when reconnecting USB, the led on adapter indicating either TD or RD (or both) activity {don't recall} will light dimly.

 

Another observation today is that with no SD card I get nothing from console now.  I think this is different from earlier.  I made no changes to on board mmc. I am fairly sure the serial adapter is ok  when this happens but I need to look at this a little deeper.

 

Hopefully all this means that I have reliable boot so long as there is only one possible source for power.  I have yet to test a backup image. Perhaps later today.

 

 

Thanks to all and I hope I have added something useful.  I will report back with more if I have something helpful.  

 

Should I start a new thread?

 

Q

 

 

Link to post
Share on other sites

Hi Quanta,

 

Depending on the type of serial connector, it may (partly) power the nanopi, I usually only connect the GND, RX and TX with my FT232RL USB to TTL Serial Adapter. I've seen cases where parts of the board didn't fully reset when the serial was still connected. You can of course start a separate thread about serial adaptor power issues, which may be a more general issue.

 

My reboot problems however also seem to occur without the serial connected, i.e. can can notice it doesn't become reachable again after reboot is invoked remotely.

 

Kind regards,

Dennis

Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...