Orangepizero does not restart any more

[FRANK333]

good morning everyone,

I installed Armbian 23.02.2 on orangepizero about a year ago without ever having a problem .
Today it suddenly stopped booting and I can't get it to start again and that's what I'm interested in, also possibly to recover internal docker configurations.
The orangepizero is powered by a 3A power supply under UPS and properly cooled, if I use the latest release on a new SD it works perfectly, so I think it is not a hardware problem.

The SD with Armbian 23.02.2 is not corrupted and I can read every part of it.
Does anyone know any tricks to get it running again?

[wulfy23]

-console connection would help ( bootlog )

-so would an earlier backup of that SD Card... ( make a backup of it now anyway )

 

personally i'd just stick with the new OS ( a year is pretty long in the tooth )... and new card... then just scp/rsync whatever is critical over to the new OS.

 

other than that try running a fsck, / seeing what the console says

 

(edit: sorry i messed the beginners bit...)

Edited May 11 by wulfy23

[FRANK333]

hi wulfy23,

the problem is that you cannot use the console because the operating system no longer starts.

If I extract the sd card and read it from the computer though, I can see all the OS files.

 

[Sune]

Do you have a USB to TTL/UART device? You would then be able to check what happens, you might even get a console through the serial connection if it starts up.

[FRANK333]

yes Sune,

 I tried with the 'serial adapter, but the console won't boot (connection parameters are correct)

 

[FRANK333]

after several attempts with the serial the boot stops like this, I think there is some problem in the boot sequence but I don't know how to unlock it.

[robertoj]

Try it with a new image, new SD, with new Linux kernel version too.

 

I have an orangepizero and it works better with Linux 6.x

[FRANK333]

@robertoj i did as you said, reinstalled latest version of armbian on new sd and i get this error when accessing via ssh:  kex_exchange_identification: read: Connection reset by peer Connection reset by....

🥵

Edited July 30 by FRANK333

[Igor]

2 hours ago, FRANK333 said:

reinstalled latest version of armbian on new sd and i get this error when accessing via ssh:

How long did you wait? Was 1st boot without interruptions? Is your SD card really good? Which image did you try?

[FRANK333]

• Yesterday evening I installed Armbian_community_24.8.0-trunk.443_Orangepizero_noble_current_6.6.36.img.xz I checked the sha , installed on sd verbatim 32gb unpacked new.
• Installed docker, portainer and container minidlna .
• First startup without a hitch worked until this morning, then the strange error, some ports remain open ;now I rebooted repaired with fsck , it starts responding to ping and after a few minutes is no longer reachable.
• I would like to try booting via SD and put the operating system on an external HDD, if it is possible.

[Igor]

1 hour ago, FRANK333 said:

6.6.36.img.xz

There are known / unknown issues with Allwinner ATM:

https://github.com/armbian/build/pull/6887

 

Use some older image from archive, not latest builds. And freeze kernel updates until this is resolved.

[robertoj]

On 7/29/2024 at 10:32 PM, FRANK333 said:

@robertoj i did as you said, reinstalled latest version of armbian on new sd and i get this error when accessing via ssh:  kex_exchange_identification: read: Connection reset by peer Connection reset by....

When you reinstall an OS in the server (or SBC in your case), the SSH will complain that it is not using the same crypto key, for cybersecurity reasons.

Usually, SHH will suggest what you do to erase the ssh key and allow you to connect with a new SSH key.

 

Use a serial-usb adapter, to test whether your orange pi zero doesnt have your original problem.

[FRANK333]

yes I reset the ssh password, but after a while it gave that error. now I have put this in for a few hours ,Armbian 24.2.1 Jammy with Linux 6.6.20-current-sunxi

[robertoj]

Frank,

Not the SSH password. The SSH KEY, which is a piece of information that the client uses to verify that the server is really the server you expect, not a new impostor.

The SSH password lets the server know that YOU are who you say you are.

 

For docker, don't try to put the whole OS in an external USB SSD. The OS in microSD card is the most reliable way to do it. Then, you can use an external USB SSD for all your Docker images and containers.

 

How was it ever working with Docker? Orange Pi Zero has 256 or 512MB RAM.

[FRANK333]

@robertoj, on the first ssh connection it asked me to delete the previous key and I did ,then it asked me for the password which I re-entered and it let me in.

As far as docker is concerned it has duned very well for a few years with portainer, minidlna,a tor relay , and deluge;managing the reserved storage it always worked then something broke , maybe due to an upgrade , and problems arose. Now I installed this old version of armbian as suggested by igor and it seems to be working fine since 24 hours.

 

[FRANK333]

I also have this problem see image below even though SD is new and armbian seems to respond well

[FRANK333]

running for about 24 hours and under stress with 100% CPU now seems to be holding up( I have fan and ceramic heatsink but temperatures are always high)🥵

[robertoj]

Add to the test:

 

"iperf3 -s" in the orange pi

 

"iperf3 -c nnn.nnn.nnn.nnn -t 0" from a PC in the LAN

[FRANK333]

I let speedtest run with iperf for about ten minutes maximum reached 82 Mibps

so far everything is still working fine , I really think the problem was caused by a botched upgrade or with some bugs in the updated packages.

 

Edited August 3 by FRANK333

[going]

22 часа назад, FRANK333 сказал:

so far everything is still working fine , I really think the problem was caused by a botched upgrade or with some bugs in the updated packages.

Which core is in the latest test (uname -r)?
What is the name of the utility shown in the screenshot?

[FRANK333]

@going armbian 24.2.1 jammy ,linux 6.6.20 current -sunxi ;utility is btop.

 

here we go again this morning returned the 'error 😵‍💫

and I can no longer access OPzero I have also tried from another host computer.

Edited August 4 by FRANK333

[Igor]

40 minutes ago, FRANK333 said:

linux 6.6.20

 

.20 is good indication, but we will try to stabilise around .43 https://github.com/armbian/build/blob/main/config/sources/families/include/sunxi64_common.inc#L33 (or higher) but we need help in debugging. Currently it is more or less only @going trying to fix this but its a lot of work. https://github.com/armbian/os?tab=readme-ov-file#latest-smoke-tests-results Smoke tests shows instability on Allwinner herd, but this is all what this primitive testing tells us. There is also another attempt bumping to 6.9 https://github.com/armbian/build/pull/6941 while we should be at 6.11.y already. That would help narrowing down this problem too.

[going]

@FRANK333 Try to install this kernel:

linux-6.6.43/linux-image-current-sunxi_24.8.0-trunk.510_armhf

linux-dtb-current-sunxi/linux-dtb-current-sunxi_24.8.0-trunk.510_armhf__6.6.43

 

1 час назад, FRANK333 сказал:

here we go again this morning returned the 'error 😵‍💫

and I can no longer access OPzero I have also tried from another host computer.

In order to analyze the situation, we need to see part of the DMESG command with an error or part of the system log with an error.

[going]

1 час назад, Igor сказал:

but its a lot of work. https://github.com/armbian/os?tab=readme-ov-file#latest-smoke-tests-results Smoke tests shows instability on Allwinner herd

Cubietruck log :

[   11.673947] brcmfmac: brcmf_fw_alloc_request: using brcm/brcmfmac43362-sdio for chip BCM43362/1
[   11.677634] brcmfmac mmc1:0001:1: Direct firmware load for brcm/brcmfmac43362-sdio.cubietech,cubietruck.bin failed with error -2
[   11.677685] brcmfmac mmc1:0001:1: Falling back to sysfs fallback for: brcm/brcmfmac43362-sdio.cubietech,cubietruck.bin
....
[   12.141976] Bluetooth: hci0: BCM20702A
[   12.142021] Bluetooth: hci0: BCM20702A1 (001.002.014) build 0000
[   12.144184] cfg80211: failed to load regulatory.db
[   12.418687] brcmfmac mmc1:0001:1: Direct firmware load for brcm/brcmfmac43362-sdio.clm_blob failed with error -2
[   12.418731] brcmfmac mmc1:0001:1: Falling back to sysfs fallback for: brcm/brcmfmac43362-sdio.clm_blob
[   12.453585] EXT4-fs (zram1): mounted filesystem 4d22a70b-63d9-448d-86f1-00d5d3260b7f r/w without journal. Quota mode: none.
[   12.457556] Bluetooth: hci0: BCM: firmware Patch file not found, tried:
[   12.457592] Bluetooth: hci0: BCM: 'brcm/BCM20702A1.cubietech,cubietruck.hcd'
[   12.457602] Bluetooth: hci0: BCM: 'brcm/BCM20702A1.hcd'
[   12.457611] Bluetooth: hci0: BCM: 'brcm/BCM.cubietech,cubietruck.hcd'
[   12.457621] Bluetooth: hci0: BCM: 'brcm/BCM.hcd'
....
[   12.756951] brcmfmac: brcmf_c_process_clm_blob: no clm_blob available (err=-2), device may have limited channels available
[   12.756988] brcmfmac: brcmf_c_process_txcap_blob: no txcap_blob available (err=-2)
[   12.757625] brcmfmac: brcmf_c_preinit_dcmds: Firmware: BCM43362/1 wl0: Apr 22 2013 14:50:00 version XXX.XXX.195.89.6 FWID 01-b30a427d
....
[   27.278196] ieee80211 phy0: brcmf_p2p_create_p2pdev: timeout occurred
[   27.278246] ieee80211 phy0: brcmf_cfg80211_add_iface: add iface p2p-dev-wlan0 type 10 failed: err=-5

I do not know what to do about it.

 

1 час назад, Igor сказал:

There is also another attempt bumping to 6.9 https://github.com/armbian/build/pull/6941 while we should be at 6.11.y already.

😁 The armbian patches contain many fixes for h616(8) chips including AC 200.

I just can't make a decision when changing the fixes. Just because I can't check how to do it right, one way or the other.

 

2 часа назад, Igor сказал:

 https://github.com/armbian/os?tab=readme-ov-file#latest-smoke-tests-results Smoke tests shows instability on Allwinner herd, but this is all what this primitive testing tells us.

Is it possible to add another column with the names of the chips so that it looks something like this:

Board name          | cpu |     Kernel version     |
--------------------|-----|------------------------|
Orange Pi Zero Plus | h5  | 6.6.43-current-sunxi64 |
--------------------|-----|------------------------|
Cubietruck          | a20 | 6.6.43-current-sunxi   |
---------------------------------------------------

And the question. Is there a message distribution service if the test contains any error?

[Igor]

42 minutes ago, going said:

I do not know what to do about it.

I think this part is safe to ignore. Wireless works, BT scans finds nothing (this perhaps needs fixing). But this hardware is too old to bother with. Its more like a reference to see if Allwinner A20 generally works.

 

43 minutes ago, going said:

The armbian patches contain many fixes for h616(8) chips including AC 200.

That is important part, yes.
 

44 minutes ago, going said:

how to do it right, one way or the other.

 

Do it always the same even not most optimal ?

 

44 minutes ago, going said:

Is it possible to add another column with the names of the chips so that it looks something like this:

We don't have this info in db at the moment.

 

45 minutes ago, going said:

Is there a message distribution service if the test contains any error?

Its binary - board failed completely, not responding but it should = error at CI level. There were few people saying that they will help to develop this test framework, but its only what I wrote some time ago. 

[FRANK333]

@going reinstalled the new kernel (trunk) and thedtb you suggested via dpkg -i everything seems to work fine (docker, etc), but if from command line I use reboot at next session I can't log in via ssh ; only if I manually reboot I can log in with sshdmesg.txtsyslog.txti should still have an armbian image from a few years ago i will install that and disable updates is the 'only way i have to reuse orangepizero.🥲