sick and tired of my Armbian desktop locking and crashing

[AxelFoley]

Is it just me .... all the time I start to use the Armbian web browser  ...including simply loading this forum the whole RockPro64 OS locks up and I have to reset,.

Typically its when I load Media web sites like the BBC ,.... but it even occurs when I load this forum!!

 

I am trying to develop some C GPIO library to help the rockPro64 community, but its hopeless. I have to work on console and web brows on a laptop.

I have ordered a KVM to help but this is really frustrating.

 

Because the HW locks up I can not detect any issues in the logs and I don't know how to set up a HW trace in the graphics drivers.

 

I have eliminated any Power issues. I have a pico cluster case and decent heatsink ... I have ordered some Heatsink fans but CPU is in the 40 Deg C.

 

Any Ideas how I find out why the HW just freezes ?

 

It must have something to do with the mali drivers ?

 

Any ideas welcome

 

 

 

[NicoD]

 

37 minutes ago, AxelFoley said:

Any Ideas how I find out why the HW just freezes ?

Do you use the Media Script from JMCC on it?
Did you try another browser? Armbian browser doesn't exist. You can try firefox or Vivaldi browser.
https://vivaldi.com/nl/blog/snapshots/vivaldi-2-3-rc-2-vivaldi-browser-snapshot-1440-37/
 

Or you using Stretch or Bionic? Did you try the other?

 

[AxelFoley]

I am using the web browser default (chromium) that came with Armbian Bionic Desktop 18.04.2 LTS  kernel 4.4.174 Armbian 7.53

 

I am trying to install the media script but the browser keeps hanging the HW\OS.

 

I will give wget\curl a try

 

[AxelFoley]

I had to transfer the script to a USB stick on a laptop then transfer it to the RockPro64 because the browser kept locking up when trying to point Chromium to the armbian media script site. I selected the defaults of System, MPV and Gstreamer from the install CLI. I then opted for the Armsoc fullscreen vsync.

 

Tonight development is a write off so I will start again tomorrow.

 

Thanks for all your help ..  fingers crossed I can stop the desktop from locking up.

Its only been this bad recently since the distro upgrades (I upgrade daily).

 

Normally the forums are fine ... and I avoid sites with media content (video always locks up) but recently even the Armbiam forum causes the OS to lock the RockPro64 OS\HW Up.

 

The install script threw errors (see attached)

 

I will do an apt autoremove, reboot

 

and report back if stability improves

install.log

[AxelFoley]

Nope .... I rebooted ... went straight into the Armbian forum using the default Chromium browser and the OS/HW locked up and froze :-(

[AxelFoley]

I have a video but its 320MB in size so it wont upload.

[NicoD]

48 minutes ago, AxelFoley said:

Nope .... I rebooted ... went straight into the Armbian forum using the default Chromium browser and the OS/HW locked up and froze :-(

So it started to do this after an upgrade?
I only asked if you were using the media script to know if that's not the problem. You didn't need to install it. But it could help, you never know. I'm not sure it works on the RockPro64.
Did you try another browser? Firefox? Vivaldi?

I know what browser crashes look like, I've had enough of them myself. I've worked with the Rock64 this week and always had them. I then tried an older kernel and it was better.
Maybe it's the same problem, or related.
@JMCC Could it be? Did you find anything?

What are the crashes like? Different kinds of crashes? Messages you get?
You could run Chromium with the terminal (just type chromium)and see if it gives any clues to what's going on.

[AxelFoley]

I have had these lockup issues  ever since I built the cluster in Dec 2018.

I initially thought it was a power issues (I was seeing some behaviour that suggested power was a problem)..

However since then I have resolved / upgraded my power source and eliminated that instability.

When I installed the base Armbian image I did follow some instructions to install some lib-mali drivers.

I suspected that may have been the issue and I associated the crashes trying to run video in a browser, so I avoided that.

However now more than ever I am seeing lockups just loading a browser and going to the armbian folder.

I will try to install firefox and see if that has similar problems.

 

 

[AxelFoley]

So far so good with firefox ....   I'll start loading media sites and see if I can replicate the crash.

If I cant in firefox I will do the same with chrome and see if its a chromium specific issue.

[AxelFoley]

Right ... I think I can recreate the problem 100% by simply loading the Armbian Forum in chromium.

I have tried for 30min to create the same OS lockup using firefox with no problems what so ever. Indeed video on sites like BBC News play flawlessly.

So it looks like the issue is specific to the default Chromium install of Armbian.

 

I had overlooked it being chromium simply because it looked like it was a HW Video Driver glitch as the entire desktop would lock up and the mouse curser would typically not even move.

 

but bingo 30seconds after I loaded Chromium ... I went straight into the Armbian Forum and it locked up on the default forum page.

 

It did not used to be this reproducible ... I avoided media sites and generally I could get on with coding befoe a crash/lock would occur.

 

I am not sure what has made the environment so much more unstable and easy to reproduce a crash as I apt update && apt upgrade several times a day.

 

is there a way I can run chromium in debug mode and get it not to delete the log files when I have to hard reboot the device ?

[AxelFoley]

Another reproducible issue is that I can get exactly the same screen lockup on boot

 

When I re installed the desktop from  the armbian-config it installed lightdm and removed nodm.

I backed out this change .. uninstalled lightdm and reinstallled nodm in case that was what was causing the instability.

 

Unfortunately in doing so ... it changed the default nodm auto login to root instead of mu default user (pico).

 

if i edit /etc/default/nodm  back to pico instead of root ..... it locks up every time in the same manner 100% every time.

 

To fix the issue I have to mount the emmc module externa,,y and edit the file back to default login as root to be able to access the desktop.

 

 

 

 

[Igor]

19 minutes ago, AxelFoley said:

default Chromium install

Armbian don't develop nor fix Chromium https://www.chromium.org/for-testers/bug-reporting-guidelines

 

12 minutes ago, AxelFoley said:

When I re installed the desktop from  the armbian-config it installed lightdm and removed nodm.

Not a bug but feature. We are about to drop nodm since bugs are not getting fixed.

[AxelFoley]

Thanks Igor ..... should I install lightdm as the default from armbian-config again ?

 

whats going to be the new default ?

[Igor]

9 minutes ago, AxelFoley said:

whats going to be the new default ?

lightdm

[AxelFoley]

armbian-config does not do a good job of installing lightdm, it failed to launch lightdm on boot with errors around trying to restart the service too quickly after it initially failed to start.

 

I don't have time to troubleshoot somebody else's sloppy mess.

I have reverted back to nodm so I can get back to writing some code.

 

The display lockup when nodm boots and logs in as any other user apart from root ... is not going to be nodm's problem I suspect.

more likely an Xorg config issue \ graphics driver.

 

good to know I can avoid 90% of my stability issues just but not using the default armbian chromium browser.

 

I am soon going to have to face facts and accept I need to reformat my entire cluster with a different distro, I just can not get Armbian stable on rockPro64.

I have been trying for 4 months. I have these lockup issues with my master node and I have 20% of my nodes constantly restarting.

I have eliminated power issues.

I have ordered a load of heatsink fans to eliminate the last thing I can think of ... heat as being the problem (despite them all having heat sinks and a pico cluster case).

 

The fans are the last thing I try before giving up on armbian and starting from scratch and loosing all my salt/cassandra/promethious/grafana cluster configuration.

[Da Alchemist]

Using Rockpro64 for Desktop Scenarios is a really bad idea at this state of development. There is no distro i would recommend for using , i have tried ayufans images, the one from mrfixit and armbian. There is a big lack of developement for this board and i really do not know why it is moved to "stable" on Armbians build system.

 

My Nanopi M4 is running really stable and smooth; acts as Desktop replacemanent since I got the SATA Hat. There are absolutely no crashes running Chromium on armbian, so i think that there are no chromium related issues.

I suspect bad hardware design on RockPro.

[AndrewDB]

4 hours ago, AxelFoley said:

Right ... I think I can recreate the problem 100% by simply loading the Armbian Forum in chromium.

In Chromium (if you absolutely insist on using it instead of Firefox), check in advanced options if hardware acceleration support is enabled or not. If it is enabled, disable it. Also check your memory/swap usage while loading the Armbian Forum in Chromium.

Finally, if your system locks up due to a kernel oops, the information is still available in dmesg, you may want to check that in a terminal, try:

tail -f /var/log/{messages,kernel,dmesg,syslog}

 

[chwe]

10 hours ago, Da Alchemist said:

There is a big lack of developement for this board and i really do not know why it is moved to "stable" on Armbians build system.

If I didn't miss something the Rockpro is still marked as wip in the downloadpage right?

 

 

but well, maybe we should rename it back to wip so that only expert=yes can build images for it.. to make it more obvious..

 

but also this sub-forum has a nice reminder:

 

My RockPi 4b was used in a pure CPU numbers crunshing project for 17 days between 75°C and 80°C without any crash.. Would I call the RockPi stable.. for sure not. It seems that it did well for this test but I've no idea if it runs stable under all the other use-cases people can imagine for the board in question.

 

Just another reminder: https://forum.armbian.com/guidelines

especially this points here:

Quote

A support question must include the following:

• Board you use
• Issue you face
• Description of your set-up (e.g. powering, connected hardware, used SD-Card)
• 'sudo armbianmonitor -u' - this gives us some needed logs which makes debugging a way easier!

 

 

12 hours ago, AxelFoley said:

I have eliminated power issues.

what makes you sure the powering issues are gone? Did you check voltage on 5v rail after replacing the PSU under stress?

[Da Alchemist]

@chwei think you understood what i wanted to say: Everywhere on Armbian Rk3399 devices are marked as WIP , only on the build-system we will find them under "stable".

 I can only compare the Rockpro64 to my Nanopi M4 an my NEO4 and I must say at this moment Rockpros performance is worse. Developement on Rockpro is done by ayufan and mrfixit, both not active members on this forum, posting Issues here does not make make much sense. There are lot crashing reports on Pines IRC channel.

 

I would not only move this Board back to the "Expert" Side, i think i would kick it completely out of Armbian, so  bad performing issues are not being related to Armbian, like it was done by @AxelFoley

[chwe]

@ayufan Last visited Saturday at 09:35 PM, it's not that he doesn't read the forum.. he just doesn't post as much as others do.. Which is understandable.. Dealing with other peoples issues isn't much fun.. especially if you can't replicate them.. The whole situation with RK3399 is a bit complicated, especially when we talk about the 4.4 kernel. Nanopis are part of the rk3399 sources with friendlyarms 4.4 fork of the Rockchip bsp kernel acts as a source (https://github.com/armbian/build/blob/f1d1ca51f32bd1a2593349ba82a1013fa55f389f/config/sources/rk3399.conf#L34-L37) whereas RockPro64 is part of the rockchip64 family with ayufans kernel as 4.4 source, but also a fork of Rockchips bsp kernel (https://github.com/armbian/build/blob/f1d1ca51f32bd1a2593349ba82a1013fa55f389f/config/sources/rockchip64.conf#L47-L49) I don't remember exactly the issue which lead to it (if I've it right in mind it's related to u-boot in the beginnings.. but maybe @martinayotte or @TonyMac32 can comment on this cause it comes from a time when I was not involved at all in rk3399 boards.. Surly this mix avoids us partly from marking rk3399 boards as stable. But surely chances are higher that a boardvendor provided/modified kernel is likely to work 'better' as an independent one.. Especially when we consider that FriendlyArm normally know what they're doing (it's not their first arm board.. )

Spoiler

They know what they're doing...

 

 

23 hours ago, Da Alchemist said:

posting Issues here does not make make much sense.

wouldn't agree on this one.. It's not that we don't fix broken kernels..   https://github.com/armbian/build/tree/master/patch IMO the most important part of armbian is related to kernelwork and get things working which don't work (well) from the sources we rely on.. But blaming a board which is marked as WIP on the downloadpage and the whole SoC family sub-forum is marked as wip as well is IMO pointless.. We do catch up issues.. that's one of the porposes of this forum.. Getting feedback what's working and what's not working to enhance support for boards. But this thread didn't show much progress.. Basically also cause:

On 4/1/2019 at 11:50 PM, chwe said:

Just another reminder: https://forum.armbian.com/guidelines

especially this points here:

Quote

A support question must include the following:

• Board you use
• Issue you face
• Description of your set-up (e.g. powering, connected hardware, used SD-Card)
• 'sudo armbianmonitor -u' - this gives us some needed logs which makes debugging a way easier!

was never answered.. A part which should actually be in the starter..

 

23 hours ago, Da Alchemist said:

I would not only move this Board back to the "Expert" Side, i think i would kick it completely out of Armbian, so  bad performing issues are not being related to Armbian, like it was done by @AxelFoley

if we would kick out every board which is a bit troublesome during wip.. We would support only a few boards here.. Just a reminder again:

Quote

RK3399 board support is WIP, "for testing only" and problems are expected! If you want to use those boards for some real cases, don't.

 

Especially if it's a platform you don't know well, just don't. Things might just not be ready.. It needs some time until they get mature.

[TonyMac32]

To be fair I am of the opinion we should not have any downloadable images for WIP targets.

Sent from my Pixel using Tapatalk

[Da Alchemist]

40 minutes ago, TonyMac32 said:

To be fair I am of the opinion we should not have any downloadable images for WIP targets.
 

Totally agree to this.

 

@chwe, think you never had a Rockpro in your hands.

53 minutes ago, chwe said:

 

if we would kick out every board which is a bit troublesome during wip.. We would support only a few boards here..

 

So Devs can concentrate on the Boards that are worth to be supported.

[pfry]

1 hour ago, TonyMac32 said:

To be fair I am of the opinion we should not have any downloadable images for WIP targets.

Whoa, I'm not. I'm a total leech on y'all's work, and I have no desire to cross-compile an image. (I prefer PC-style native installations like LFS and Gentoo - from an end-user perspective Armbian is the least-customized distro that I use.) I've had no issues with my NanoPC-T4, and the super-easy eMMC+NVMe install is unique (afaik). Can't beat it with a stick. I can't contribute much, as all I have in excess is Internet bandwidth and old PC hardware (and somewhat off-the-rails forum posts).

[martinayotte]

2 hours ago, TonyMac32 said:

To be fair I am of the opinion we should not have any downloadable images for WIP targets.

Euuh ? I've just turned it ON for PineH64 ... I will turn it OFF tomorrow ...

(That was because of confidence of @NicoD  )

 

[Da Alchemist]

@martinayotte, pineH64 is working better than Rockpro 

[martinayotte]

Just now, Da Alchemist said:

pineH64 is working better than Rockpro 

Right ! at least for the "reset/reboot" question, and it has 3GB RAM ... I will leave @NicoD to provide his testimonies ...

[Da Alchemist]

@martinayotte, i do not have any reboot issues on my pineh64 A  running OMV ,PIhole and Logitechmediaserver on it without any problems.

[martinayotte]

3 minutes ago, Da Alchemist said:

i do not have any reboot issues on my pineh64 A

Right ! But all other H6 have the issue, and I'm struggling since days to figure out why ...

[NicoD]

Boy, this has gotten off topic 
 

2 hours ago, TonyMac32 said:

To be fair I am of the opinion we should not have any downloadable images for WIP targets.

Hard to agree with this situation of the NanoPi M4/T4/NEO4 still being in WIP. 
I don't have a RockPro64, so I can't judge on that. 
I do have many other boards. And I know that it's better to have an image with faults than to have nothing at all. 
 

 

15 minutes ago, martinayotte said:

Euuh ? I've just turn it ON for PineH64 ... I will turn it OFF tomorrow ...

(That was because of confidence of @NicoD  )

 

6 minutes ago, martinayotte said:

Right ! at least for the "reset/reboot" question, and it has 3GB RAM ... I will leave @NicoD to provide his testimonies ...

Reset works, 3GB also. Display resolution is set right, wifi works from first boot, ... 
You just can't choose 1.8Ghz. But it is actually better to run it at 1.5Ghz. Maybe an idea for the OPi3, but keep 1.8Ghz available. I know most people will not use a fan on it, so it's not useful to leave it at 1.8Ghz. 

 

13 minutes ago, Da Alchemist said:

@martinayotte, pineH64 is working better than Rockpro 

I was surprised. I kept it on the side for after the Orange Pi 3. And expected it to be worse. It runs great barebones. It isn't going to break performance records, but it behaves nice.
 

[TonyMac32]

27 minutes ago, NicoD said:

Hard to agree with this situation of the NanoPi M4/T4/NEO4 still being in WIP. 

 

RK3399 support is a total and utter mess.  When we finally fix it, it will break the upgrade path on very nearly every board, since our current patching/boot situation is impossibly convoluted.  Given the OP is not alone, and I completely understand his frustration, just imagine when that happens.  

 

51 minutes ago, martinayotte said:

Euuh ? I've just turned it ON for PineH64 ... I will turn it OFF tomorrow ...

Well, I did say "opinion".  I would guess it takes more than my automotive-engineering-minded release conservatism to shut down or change the direction of the entire project.  I will open a new thread on this topic, I may have a compromise in mind. 

[chwe]

1 hour ago, Da Alchemist said:

 

@chwe, think you never had a Rockpro in your hands.

2 hours ago, chwe said:

 

if we would kick out every board which is a bit troublesome during wip.. We would support only a few boards here..

 

So Devs can concentrate on the Boards that are worth to be supported.

forget to state but yes, I never had one.. but still considering buying one.. Devs here normally focus on board they're interested... not boards someone wants to have up..

 

18 minutes ago, Da Alchemist said:

@martinayotte, pineH64 is working better than Rockpro 

well.. send me rockpro and I'll send you a pineH64 back..

 

39 minutes ago, TonyMac32 said:

it will break the upgrade path on very nearly every board, since our current patching/boot situation is impossibly convoluted.

wouldn't agree on this.. but for sure it needs some sort of a merge script to ensure paths are properly set.. or at least old installations won't get updated into the merged family..

1 hour ago, TonyMac32 said:

Given the OP is not alone, and I completely understand his frustration, just imagine when that happens.  

given all boards are marked wip stated clearly that you shouldn't use them productive... Means you shouldn't use them productive..

 

1 hour ago, TonyMac32 said:

Well, I did say "opinion".  I would guess it takes more than my automotive-engineering-minded release conservatism to shut down or change the direction of the entire project.  I will open a new thread on this topic, I may have a compromise in mind. 

good idea.. even when I don't fully agree on the no wip image thing.