Jump to content

Random reboots on NanoPC-T4 with Armbian 5.69

Carlos Eduardo De Paula

By Carlos Eduardo De Paula
in Rockchip

 Share

Recommended Posts

Carlos Eduardo De Paula

Carlos Eduardo De Paula

Posted
Posted (edited)

As I reported here (https://github.com/armbian/build/issues/1230), I'm seeing frequent reboots and Kernel Panics on a RK3399 board while running ElasticSearch, a kinda heavy application that runs on Java.

 

I currently run a Kubernetes cluster on two RK3399 boards (NanoPC-T4). One of them I has a version of DietPi with a Kernel 4.4.138 compiled by myself. and runs perfectly. The other had a newer version of a Kernel (4.4.154) also compiled by me and that's where I started seeing the reboots.

 

After this, I installed Armbian thinking it might be something with my DietPi distro. I installed 5.69 with Kernel 4.4.167 and saw the same problems. I did a full memory test with *memtester* and found no errors.

 

After I reverted this board to the same 4.4.138 and it's DTB, the board is stable again. I can provide the DTB and Kernel I used that works. Looks like it's something from either newer kernels or newer DTB. I fetched the DTBs from FriendlyArm own tree (https://github.com/friendlyarm/kernel-rockchip) but compiled into the Rockchip Kernel (https://github.com/rockchip-linux/kernel). \

 

My board info:

  

_   _                   ____   ____   _____ _  _   
| \ | | __ _ _ __   ___ |  _ \ / ___| |_   _| || |  
|  \| |/ _` | '_ \ / _ \| |_) | |       | | | || |_ 
| |\  | (_| | | | | (_) |  __/| |___    | | |__   _|
|_| \_|\__,_|_| |_|\___/|_|    \____|   |_|    |_|  
                                                    

Welcome to ARMBIAN 5.69 stable Debian GNU/Linux 9 (stretch) 4.4.167-rk3399   
System load:   2.93 1.25 0.47   Up time:       1 min
Memory usage:  44 % of 3811MB   IP:            192.168.1.72 10.42.0.0
CPU temp:      56°C           
Usage of /:    25% of 15G    

[ 0 security updates available, 15 updates total: apt upgrade ]
Last check: 2019-01-17 16:11

[ General system configuration (beta): armbian-config ]

$ uname -a
Linux k8s-armnode2 4.4.167-rk3399 #70 SMP Thu Jan 10 20:46:25 CET 2019 aarch64 GNU/Linux

 

 

Here are the dumps I got from two crashes:

 

1:

  Reveal hidden contents

 

2:

  Reveal hidden contents

I can provide more info if needed.

Edited by Igor
spoilers
littlema

littlema

Posted
Posted (edited)
  On 1/18/2019 at 7:37 PM, Carlos Eduardo De Paula said:

As I reported here (https://github.com/armbian/build/issues/1230), I'm seeing frequent reboots and Kernel Panics on a RK3399 board while running ElasticSearch, a kinda heavy application that runs on Java.

 

I currently run a Kubernetes cluster on two RK3399 boards (NanoPC-T4). One of them I has a version of DietPi with a Kernel 4.4.138 compiled by myself. and runs perfectly. The other had a newer version of a Kernel (4.4.154) also compiled by me and that's where I started seeing the reboots.

 

After this, I installed Armbian thinking it might be something with my DietPi distro. I installed 5.69 with Kernel 4.4.167 and saw the same problems. I did a full memory test with *memtester* and found no errors.

 

After I reverted this board to the same 4.4.138 and it's DTB, the board is stable again. I can provide the DTB and Kernel I used that works. Looks like it's something from either newer kernels or newer DTB. I fetched the DTBs from FriendlyArm own tree (https://github.com/friendlyarm/kernel-rockchip) but compiled into the Rockchip Kernel (https://github.com/rockchip-linux/kernel). \

 

My board info:

  

_   _                   ____   ____   _____ _  _   
| \ | | __ _ _ __   ___ |  _ \ / ___| |_   _| || |  
|  \| |/ _` | '_ \ / _ \| |_) | |       | | | || |_ 
| |\  | (_| | | | | (_) |  __/| |___    | | |__   _|
|_| \_|\__,_|_| |_|\___/|_|    \____|   |_|    |_|  
                                                    

Welcome to ARMBIAN 5.69 stable Debian GNU/Linux 9 (stretch) 4.4.167-rk3399   
System load:   2.93 1.25 0.47   Up time:       1 min
Memory usage:  44 % of 3811MB   IP:            192.168.1.72 10.42.0.0
CPU temp:      56°C           
Usage of /:    25% of 15G    

[ 0 security updates available, 15 updates total: apt upgrade ]
Last check: 2019-01-17 16:11

[ General system configuration (beta): armbian-config ]

$ uname -a
Linux k8s-armnode2 4.4.167-rk3399 #70 SMP Thu Jan 10 20:46:25 CET 2019 aarch64 GNU/Linux

 

 

Here are the dumps I got from two crashes:

 

1:

  Reveal hidden contents

 

2:

  Reveal hidden contents

I can provide more info if needed.

Expand  

I have the same problem with you,and my network crashed when I download bittorrent files.

Edited by Igor
spoilers
Carlos Eduardo De Paula

Carlos Eduardo De Paula

Posted
  • Author
Posted

Sorry about the delay answering your message @littlema, I uploaded the Kernel files I tried and got crashes.

 

I tried with latest sources and patches from Armbian too but same results. I only see these crashes while running ElasticSearch.

 

This is the last crash I got with the Kernel 4.4.174 built from Armbian builder. I was running strace on the Java process:

 

  Reveal hidden contents

 

https://ufile.io/rwsz1

 

If anyone have a clue on how to proceed, please let me know.

Tido

Tido

Posted
Posted
  On 1/18/2019 at 7:37 PM, Carlos Eduardo De Paula said:

I'm seeing frequent reboots and Kernel Panics

Expand  

This can also be a power problem.

How many devices do you have attached?  (can you put a power USB-Hub in between for example)

How many Watt can you PowerSupply deliver?

Igor

Igor

Posted
Posted

RK3399 support is from our perspective declared as WIP, "for testing only" and such problems are expected! If you want to use those boards for some real cases, don't. Not now. I would like to tell you hove to fix those problems, but I don't know. Not without deep investigation ... which is sadly very far away due to huge demand on our tiny resources from users side.

 

In case of T4/M4/Neo4 we use more or less stock kernel from Friendlyarm which means if they will fix this in the mean time, it will show up in Armbian at once.

Carlos Eduardo De Paula

Carlos Eduardo De Paula

Posted
  • Author
Posted

On friday, I ended up erasing all flash and re-flashing the image I had working on another board to it. What is weird is that the Kernel and the DTB are the same, what's different is that I built a newer u-boot and created a kernel partition that is mountable so I could configure u-boot menu via extlinux.conf file.

 

The board is stable since then so I think it might be something wrong either with u-boot or the partition scheme I use. Question, can u-boot influence the stability of the OS? I have no clue.

 

Thanks!

Guest
This topic is now closed to further replies.
 Share
Go to topic listing
×
×
  • Create New...

Important Information

Terms of Use - Privacy Policy - Guidelines

  I accept