Helios64 Support

[gprovost]

This main thread has been locked. Thanks to the Armbian team, we now have our own Kobol club (kinda sub forum) dedicated to our products (Helios64 & Helios4).

So now you are encourage to create individual threads for new issue, discuss feature, share setup, etc...

This will help to improve overall readability.

 

Note : Before asking a question please insure the information is not already available on Helios64 Wiki or addressed in a previous post in this forum.

 

 

Latest Build :

 

Check : https://dl.armbian.com/helios64/

 

Archived Build :

 

Check : https://archive.armbian.com/helios64/archive/

 

Known Issues :

 

Work-in-Progress images, not all board features are supported yet.

 

Find software support status on our wiki or  our here on the forum

 

 

[aprayoga]

On 10/15/2020 at 2:11 PM, RaSca said:

Hi everybody,

as I mentioned on the helios64 site, there seems to be a problem with external USB drives.

I've tried with two different of them that works perfectly on other systems, but not here.
The first one, once connected (it doesn't matter front or rear), gives these messages:

 

Oct 13 12:42:18 helios64 kernel: usb 2-1.3: USB disconnect, device number 8
Oct 13 12:42:21 helios64 kernel: usb 2-1.2: new SuperSpeed Gen 1 USB device number 9 using xhci-hcd
Oct 13 12:42:31 helios64 kernel: usb 2-1.2: New USB device found, idVendor=174c, idProduct=5106, bcdDevice= 0.01
Oct 13 12:42:31 helios64 kernel: usb 2-1.2: New USB device strings: Mfr=2, Product=3, SerialNumber=1
Oct 13 12:42:31 helios64 kernel: usb 2-1.2: Product: AS2105
Oct 13 12:42:31 helios64 kernel: usb 2-1.2: Manufacturer: ASMedia
Oct 13 12:42:37 helios64 kernel: usb 2-1.2: can't set config #1, error -110

The second one:
 

Oct 14 15:41:07 helios64 kernel: usb 1-1.3: new high-speed USB device number 55 using xhci-hcd
Oct 14 15:41:07 helios64 kernel: usb 1-1.3: New USB device found, idVendor=1e68, idProduct=003a, bcdDevice= 1.32
Oct 14 15:41:07 helios64 kernel: usb 1-1.3: New USB device strings: Mfr=1, Product=2, SerialNumber=3
Oct 14 15:41:07 helios64 kernel: usb 1-1.3: Product: DS pocket light
Oct 14 15:41:07 helios64 kernel: usb 1-1.3: Manufacturer: TrekStor
Oct 14 15:41:07 helios64 kernel: usb 1-1.3: SerialNumber: 201108241936
Oct 14 15:41:07 helios64 kernel: usb-storage 1-1.3:1.0: USB Mass Storage device detected
Oct 14 15:41:07 helios64 kernel: scsi host5: usb-storage 1-1.3:1.0
Oct 14 15:41:07 helios64 kernel: usb 1-1.3: USB disconnect, device number 55
Oct 14 15:41:07 helios64 kernel: usb 1-1.3: new high-speed USB device number 56 using xhci-hcd
Oct 14 15:41:08 helios64 kernel: usb 1-1.3: device descriptor read/all, error -71
Oct 14 15:41:08 helios64 kernel: usb 1-1.3: new high-speed USB device number 57 using xhci-hcd
Oct 14 15:41:09 helios64 kernel: usb 1-1.3: device descriptor read/64, error -71
Oct 14 15:41:09 helios64 kernel: usb 1-1.3: Device not responding to setup address.
Oct 14 15:41:10 helios64 kernel: usb 1-1.3: Device not responding to setup address.

 

 

Any suggestion?

Could you try edit the /boot/armbianEnv.txt and add idVendor and idProduct of your drive to usbstoragequirks ?

usbstoragequirks=0x2537:0x1066:u,0x2537:0x1068:u,0x174c:0x5106:u,0x1e68:0x003a:u

reboot the system and see whether your external USB drive can be recognized.

 

[ebin-dev]

5 hours ago, aprayoga said:

Ah i just remember, do you have RTC battery or the UPS battery? the batteries is needed to make auto power on working correctly.

More info will be described on the wiki.

@aprayoga Thanks for your reply. Your guess was just right - I have temporarily disconnected the UPS battery before I was experimenting with various system configurations so that I would be able to recover from a hanging system by disconnecting the main power.

 

If you are improving the wiki pages - there are actually at least three topics of interest:

 

1.) how to enable "auto power on"

2.) how to use the RTC to automatically start the system at predefined times

3.) how to control powering hdd rails A and B

[raoulh]

Hi,

 

I tried different version, but the result are the same. I can't get correct speed with ethernet, be it the 2.5G or the 1G.

 

I installed armbian focal 20.08.10. The helio64 is connected through a 10G switch on eth1, and on another 1g switch on eth0.

 

I did some iperf  tests and this is what I'm getting for both connections:

 

This is with eth0

➜  ~ iperf3 -c 192.168.0.86 -p 12345  
Connecting to host 192.168.0.86, port 12345
[  5] local 192.168.0.238 port 58950 connected to 192.168.0.86 port 12345
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  67.4 MBytes   565 Mbits/sec  611   17.0 KBytes        
[  5]   1.00-2.00   sec  68.0 MBytes   571 Mbits/sec  606   18.4 KBytes        
[  5]   2.00-3.00   sec  69.6 MBytes   584 Mbits/sec  633   18.4 KBytes        
[  5]   3.00-4.00   sec  69.3 MBytes   582 Mbits/sec  604   19.8 KBytes        
[  5]   4.00-5.00   sec  70.0 MBytes   587 Mbits/sec  589   19.8 KBytes        
[  5]   5.00-6.00   sec  71.3 MBytes   598 Mbits/sec  613   19.8 KBytes        
[  5]   6.00-7.00   sec  70.5 MBytes   591 Mbits/sec  570   19.8 KBytes        
[  5]   7.00-8.00   sec  67.8 MBytes   569 Mbits/sec  530   17.0 KBytes        
[  5]   8.00-9.00   sec  69.9 MBytes   586 Mbits/sec  580   32.5 KBytes        
[  5]   9.00-10.00  sec  69.4 MBytes   582 Mbits/sec  561   15.6 KBytes        
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   693 MBytes   582 Mbits/sec  5897             sender
[  5]   0.00-10.00  sec   693 MBytes   581 Mbits/sec                  receiver

 

And this for eth1

➜  ~ iperf3 -c 192.168.0.87 -p 12345
Connecting to host 192.168.0.87, port 12345
[  5] local 192.168.0.238 port 51834 connected to 192.168.0.87 port 12345
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  65.6 MBytes   550 Mbits/sec  566   17.0 KBytes        
[  5]   1.00-2.00   sec  69.0 MBytes   579 Mbits/sec  613   18.4 KBytes        
[  5]   2.00-3.00   sec  69.3 MBytes   582 Mbits/sec  620   19.8 KBytes        
[  5]   3.00-4.00   sec  69.5 MBytes   583 Mbits/sec  617   18.4 KBytes        
[  5]   4.00-5.00   sec  69.2 MBytes   581 Mbits/sec  623   21.2 KBytes        
[  5]   5.00-6.00   sec  63.0 MBytes   529 Mbits/sec  460   17.0 KBytes        
[  5]   6.00-7.00   sec  66.7 MBytes   559 Mbits/sec  521   19.8 KBytes        
[  5]   7.00-8.00   sec  68.5 MBytes   575 Mbits/sec  559   29.7 KBytes        
[  5]   8.00-9.00   sec  70.0 MBytes   587 Mbits/sec  565   26.9 KBytes        
[  5]   9.00-10.00  sec  71.2 MBytes   597 Mbits/sec  563   19.8 KBytes        
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   682 MBytes   572 Mbits/sec  5707             sender
[  5]   0.00-10.00  sec   682 MBytes   572 Mbits/sec                  receiver

Something is wrong here... Any idea of the cause of not getting full bandwidth?

 

Another unrelated issue I had is I wanted to use the helios64 as a iSCSI target and wanted to configure LIO. The problem is that the armbian kernel config does not have those options enabled as modules:

CONFIG_TARGET_CORE=m
CONFIG_TCM_IBLOCK=m
CONFIG_TCM_FILEIO=m
CONFIG_TCM_PSCSI=m
CONFIG_TCM_USER2=m

I did rebuild an armbian kernel with those enabled, and I can get iSCSI target working on the helios64.

 

Could it be possible to include those options for the next armbian release? I think having iSCSI is a must have for helios64.

 

Thanks
 

[gprovost]

2 minutes ago, raoulh said:

I tried different version, but the result are the same. I can't get correct speed with ethernet, be it the 2.5G or the 1G.

 

I installed armbian focal 20.08.10. The helio64 is connected through a 10G switch on eth1, and on another 1g switch on eth0.

 

We will do an announcement soon but we realized we made a design mistake on the Helios64 2.5GbE port (LAN2 | eth1) which makes it not perform well in 1000Mb/s mode, however no impact in 2500baseT mode.

 

Regarding your iperf test, seems a bit funny. You sure your routing table is correct for traffic to go really to the expected interface since you are on the same subnet.

Can you do the same tests, but for each test disconnect the cable of the interface you are not testing.... or at least use different subnet.

Also can you check interface speed with ethtool (ethtool eth0 and ethtool eth1).

[flower]

18 minutes ago, gprovost said:

 

We will do an announcement soon but we realized we made a design mistake on the Helios64 2.5GbE port (LAN2 | eth1) which makes it not perform well in 1000Mb/s mode, however no impact in 2500baseT mode.

Does this mean i can enable tx offloading again? 

 

It is connected to a 2.5gbe switch

[gprovost]

7 minutes ago, flower said:

Does this mean i can enable tx offloading again? 

 

No this is not related to tx offloading. For now we still don't know why tx offloading needs to be disabled on LK5.8, while on LK4.4 it works without issue.

[RaSca]

2 hours ago, aprayoga said:

Could you try edit the /boot/armbianEnv.txt and add idVendor and idProduct of your drive to usbstoragequirks ?

usbstoragequirks=0x2537:0x1066:u,0x2537:0x1068:u,0x174c:0x5106:u,0x1e68:0x003a:u

reboot the system and see whether your external USB drive can be recognized.

 

Hey aprayoga, thanks for your answer, I would have answered earlier, but it seems that I can't post more than ONE (!) message per day on this forum, we need to find a better way to interact with users, because it is hard to both follow discussions and search in this one single immense thread.

BTW, following the fact that with a powered USB HUB I was able to see both disks, I took a cable with the double end (one for data and the other one for the power), connected both to the two back ports, and both disks are now usable. It is in any case strange, because I can connect these disks with just the single data cable to any raspberry and work on them, but for me this is an acceptable workaround, instead of playing with the /boot/armbianEnv.txt boot parameters.

[Werner]

11 minutes ago, RaSca said:

but it seems that I can't post more than ONE (!) message per day on this forum,

This restriction applies to all new users. You can thank all the spamers out there which try to abuse forums to spread their BS.

You received a like which will lift the restrictions for you either immediately or after next cron batch. Not sure.

[Borromini]

@flower and @barnumbirr Which Noctua fans did you get?

 

Also curious if anyone has installed a dust filter behind the front grille.

 

Thanks!

[antsu]

4 hours ago, RamsDeep said:

Is there a How-To for deploying ZFS anywhere?

I am a beginner and figuring this out as I go.

Installing the module from OMV does not appear to work for me currently

 

Thanks

Use kernel 5.8 and follow the instructions on the thread above.

[flower]

For reference: this is my test on 2.5Gbe. One switch. helios64 has much load atm though.

tx and rx offloading is off

 

Server listening on 9999
-----------------------------------------------------------
Accepted connection from 192.168.178.10, port 47294
[  5] local 192.168.178.2 port 9999 connected to 192.168.178.10 port 47296
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec   211 MBytes  1.77 Gbits/sec                  
[  5]   1.00-2.00   sec   190 MBytes  1.60 Gbits/sec                  
[  5]   2.00-3.00   sec   220 MBytes  1.85 Gbits/sec                  
[  5]   3.00-4.00   sec   167 MBytes  1.40 Gbits/sec                  
[  5]   4.00-5.00   sec   226 MBytes  1.89 Gbits/sec                  
[  5]   5.00-6.00   sec   211 MBytes  1.77 Gbits/sec                  
[  5]   6.00-7.00   sec   217 MBytes  1.82 Gbits/sec                  
[  5]   7.00-8.00   sec   201 MBytes  1.69 Gbits/sec                  
[  5]   8.00-9.00   sec   271 MBytes  2.28 Gbits/sec                  
[  5]   9.00-10.00  sec   215 MBytes  1.81 Gbits/sec                  
[  5]  10.00-10.00  sec   256 KBytes  1.25 Gbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-10.00  sec  2.08 GBytes  1.79 Gbits/sec                  receiver

 

[barnumbirr]

1 hour ago, Borromini said:

@flower and @barnumbirr Which Noctua fans did you get?

 

Also curious if anyone has installed a dust filter behind the front grille.

 

Thanks!

Just the standard Noctua NF-A8 PWM (Chromax.Black.swap because I'm a weeb :D). No dust filter for me yet: I think it might impede airflow too much but I'll see over time.

[flower]

1 hour ago, Borromini said:

@flower and @barnumbirr Which Noctua fans did you get?

 

Also curious if anyone has installed a dust filter behind the front grille.

 

Thanks!

80mm 2200rpm pwm

No i dont have a dust filter. 

 

This one https://www.heise.de/preisvergleich/noctua-nf-a8-pwm-a1165563.html

[inconsistant]

Ran an update and now on 20.08.13 but getting errors immediately. Thought maybe it was something I had done so was preparing to start with a fresh install. First boot on SD with 20.08.13 gets the following error on boot.

 

Spoiler

[   26.679038] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP
[   26.679536] Modules linked in: r8152 snd_soc_hdmi_codec hantro_vpu(C) rockchi                        p_vdec(C) snd_soc_rockchip_i2s rockchipdrm v4l2_h264 rockchip_rga videobuf2_dma_                        contig dw_mipi_dsi snd_soc_core zstd videobuf2_vmalloc videobuf2_dma_sg v4l2_mem                        2mem dw_hdmi videobuf2_memops snd_pcm_dmaengine analogix_dp leds_pwm snd_pcm vid                        eobuf2_v4l2 drm_kms_helper panfrost videobuf2_common snd_timer cec gpio_charger                         videodev gpu_sched rc_core snd pwm_fan sg soundcore fusb30x(C) mc drm drm_panel_                        orientation_quirks gpio_beeper cpufreq_dt zram lm75 ip_tables x_tables autofs4 r                        aid10 raid1 raid0 multipath linear raid456 async_raid6_recov async_memcpy async_                        pq async_xor async_tx realtek md_mod dwmac_rk stmmac_platform stmmac mdio_xpcs a                        dc_keys
[   26.685159] CPU: 5 PID: 45 Comm: kworker/5:1 Tainted: G         C        5.8.                        16-rockchip64 #20.08.13
[   26.685963] Hardware name: Helios64 (DT)
[   26.686330] Workqueue: events dbs_work_handler
[   26.686730] pstate: 00000085 (nzcv daIf -PAN -UAO BTYPE=--)
[   26.687228] pc : do_undefinstr+0x2ec/0x310
[   26.687594] lr : do_undefinstr+0x1e0/0x310
[   26.687959] sp : ffff800011d6b320
[   26.688256] x29: ffff800011d6b320 x28: ffff0000f66a1d00
[   26.688730] x27: ffff0000f588ecf8 x26: ffff0000f66a1d00
[   26.689202] x25: ffff8000119cf3a8 x24: 0000000001000001
[   26.689675] x23: 0000000080000085 x22: ffff8000100101f8
[   26.690148] x21: ffff800011d6b4d0 x20: ffff0000f66a1d00
[   26.690620] x19: ffff800011d6b390 x18: 0000000000000000
[   26.691092] x17: 0000000000000000 x16: 0000000000000000
[   26.691563] x15: 0000000000000000 x14: 0000000000000137
[   26.692036] x13: 0000000000000355 x12: 000000000000035d
[   26.692507] x11: 0000000000000001 x10: 0000000000000a20
[   26.692979] x9 : ffff800011d6b460 x8 : ffff0000f66a2780
[   26.693450] x7 : 0000000000000001 x6 : ffff800011d6b378
[   26.693922] x5 : 00000000d5300000 x4 : ffff800011806118
[   26.694394] x3 : 0000000092800000 x2 : 0000000000000002
[   26.694866] x1 : ffff0000f66a1d00 x0 : 0000000080000085
[   26.695338] Call trace:
[   26.695563]  do_undefinstr+0x2ec/0x310
[   26.695903]  el1_sync_handler+0x88/0x110
[   26.696255]  el1_sync+0x7c/0x100
[   26.696548]  unwind_frame+0x78/0x1a0
[   26.696869]  return_address+0x58/0x90
[   26.697200]  preempt_count_add+0xb8/0x158
[   26.697564]  _raw_spin_lock_irqsave+0x28/0xa0
[   26.697954]  prepare_to_wait_event+0x24/0xe8
[   26.698339]  rk3x_i2c_xfer+0x2c8/0x448
[   26.698675]  __i2c_transfer+0x144/0x650
[   26.699018]  i2c_transfer+0x60/0x128
[   26.699338]  i2c_transfer_buffer_flags+0x5c/0x88
[   26.699752]  regmap_i2c_write+0x20/0x58
[   26.700099]  _regmap_raw_write_impl+0x6f8/0x8c0
[   26.700504]  _regmap_bus_raw_write+0x68/0x88
[   26.700886]  _regmap_write+0x6c/0x160
[   26.701216]  _regmap_update_bits+0xf8/0x110
[   26.701591]  regmap_update_bits_base+0x64/0x98
[   26.701990]  regulator_set_voltage_sel_regmap+0x4c/0x98
[   26.702458]  _regulator_call_set_voltage_sel+0x78/0xd0
[   26.702916]  _regulator_do_set_voltage+0x474/0x5f0
[   26.703343]  regulator_set_voltage_rdev+0xac/0x230
[   26.703771]  regulator_do_balance_voltage+0x280/0x420
[   26.704223]  regulator_balance_voltage+0x50/0x90
[   26.704634]  regulator_set_voltage_unlocked+0x94/0x118
[   26.705090]  regulator_set_voltage+0x50/0x90
[   26.705474]  _set_opp_voltage+0x44/0x110
[   26.705826]  dev_pm_opp_set_rate+0x250/0x618
[   26.706213]  set_target+0x40/0x88 [cpufreq_dt]
[   26.706611]  __cpufreq_driver_target+0x2c4/0x668
[   26.707023]  od_dbs_update+0xbc/0x1a0
[   26.707353]  dbs_work_handler+0x40/0x80
[   26.707700]  process_one_work+0x1c4/0x470
[   26.708060]  worker_thread+0x4c/0x420
[   26.708391]  kthread+0x118/0x150
[   26.708682]  ret_from_fork+0x10/0x34
[   26.709006] Code: f9401bf7 17ffff7d a9025bf5 f9001bf7 (d4210000)
[   26.709549] ---[ end trace 96ce5fbef0a4e2d9 ]---
[   26.709961] note: kworker/5:1[45] exited with preempt_count 2
[   26.710629] ------------[ cut here ]------------
[   26.711057] WARNING: CPU: 5 PID: 0 at kernel/rcu/tree.c:618 rcu_eqs_enter.isr                        a.0+0x150/0x168
[   26.711802] Modules linked in: r8152 snd_soc_hdmi_codec hantro_vpu(C) rockchi                        p_vdec(C) snd_soc_rockchip_i2s rockchipdrm v4l2_h264 rockchip_rga videobuf2_dma_                        contig dw_mipi_dsi snd_soc_core zstd videobuf2_vmalloc videobuf2_dma_sg v4l2_mem                        2mem dw_hdmi videobuf2_memops snd_pcm_dmaengine analogix_dp leds_pwm snd_pcm vid                        eobuf2_v4l2 drm_kms_helper panfrost videobuf2_common snd_timer cec gpio_charger                         videodev gpu_sched rc_core snd pwm_fan sg soundcore fusb30x(C) mc drm drm_panel_                        orientation_quirks gpio_beeper cpufreq_dt zram lm75 ip_tables x_tables autofs4 r                        aid10 raid1 raid0 multipath linear raid456 async_raid6_recov async_memcpy async_                        pq async_xor async_tx realtek md_mod dwmac_rk stmmac_platform stmmac mdio_xpcs a                        dc_keys
[   26.717407] CPU: 5 PID: 0 Comm: swapper/5 Tainted: G      D  C        5.8.16-                        rockchip64 #20.08.13
[   26.718188] Hardware name: Helios64 (DT)
[   26.718541] pstate: 200003c5 (nzCv DAIF -PAN -UAO BTYPE=--)
[   26.719037] pc : rcu_eqs_enter.isra.0+0x150/0x168
[   26.719457] lr : rcu_eqs_enter.isra.0+0x1c/0x168
[   26.719867] sp : ffff800011c5bf00
[   26.720164] x29: ffff800011c5bf00 x28: 0000000000000000
[   26.720636] x27: ffff0000f6ea6580 x26: 0000000000000000
[   26.721109] x25: 0000000000000000 x24: ffff80001125f1f8
[   26.721581] x23: ffff0000f6ea6580 x22: ffff8000114f2738
[   26.722053] x21: ffff8000117fa2a0 x20: ffff8000117f9980
[   26.722525] x19: ffff8000114f47c0 x18: 000000000000000e
[   26.722997] x17: 0000000000000001 x16: 0000000000000019
[   26.723468] x15: 0000000000000004 x14: 0000000000000149
[   26.723940] x13: 0000000000000003 x12: 0000000000000000
[   26.724412] x11: 0000000000000040 x10: ffff800011819518
[   26.724884] x9 : ffff800011819510 x8 : ffff0000f6800028
[   26.725356] x7 : 0000000000000000 x6 : 000000003be2522a
[   26.725832] x5 : 00ffffffffffffff x4 : 0000000000002774
[   26.726304] x3 : ffff8000114de018 x2 : 4000000000000000
[   26.726776] x1 : 4000000000000002 x0 : ffff0000f77c87c0
[   26.727248] Call trace:
[   26.727472]  rcu_eqs_enter.isra.0+0x150/0x168
[   26.727864]  rcu_idle_enter+0x10/0x20
[   26.728193]  do_idle+0x20c/0x288
[   26.728484]  cpu_startup_entry+0x28/0x68
[   26.728837]  secondary_start_kernel+0x140/0x178
[   26.729240] ---[ end trace 96ce5fbef0a4e2da ]---
[   36.086089] EXT4-fs (mmcblk0p1): resized to 3670016 blocks
 

 

[aprayoga]

22 hours ago, ebin-dev said:

3.) how to control powering hdd rails A and B

I forgot to answer this. the power staggering is handled by bootloader and not user configurable.
We have tested various HDD brand/model and the highest current during spin up can go up to 7 seconds. We are considering the delay to be user configurable in future.

 

10 hours ago, inconsistant said:

Ran an update and now on 20.08.13 but getting errors immediately. Thought maybe it was something I had done so was preparing to start with a fresh install. First boot on SD with 20.08.13 gets the following error on boot.

 

  Reveal hidden contents

[   26.679038] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP
[   26.679536] Modules linked in: r8152 snd_soc_hdmi_codec hantro_vpu(C) rockchi                        p_vdec(C) snd_soc_rockchip_i2s rockchipdrm v4l2_h264 rockchip_rga videobuf2_dma_                        contig dw_mipi_dsi snd_soc_core zstd videobuf2_vmalloc videobuf2_dma_sg v4l2_mem                        2mem dw_hdmi videobuf2_memops snd_pcm_dmaengine analogix_dp leds_pwm snd_pcm vid                        eobuf2_v4l2 drm_kms_helper panfrost videobuf2_common snd_timer cec gpio_charger                         videodev gpu_sched rc_core snd pwm_fan sg soundcore fusb30x(C) mc drm drm_panel_                        orientation_quirks gpio_beeper cpufreq_dt zram lm75 ip_tables x_tables autofs4 r                        aid10 raid1 raid0 multipath linear raid456 async_raid6_recov async_memcpy async_                        pq async_xor async_tx realtek md_mod dwmac_rk stmmac_platform stmmac mdio_xpcs a                        dc_keys
[   26.685159] CPU: 5 PID: 45 Comm: kworker/5:1 Tainted: G         C        5.8.                        16-rockchip64 #20.08.13
[   26.685963] Hardware name: Helios64 (DT)
[   26.686330] Workqueue: events dbs_work_handler
[   26.686730] pstate: 00000085 (nzcv daIf -PAN -UAO BTYPE=--)
[   26.687228] pc : do_undefinstr+0x2ec/0x310
[   26.687594] lr : do_undefinstr+0x1e0/0x310
[   26.687959] sp : ffff800011d6b320
[   26.688256] x29: ffff800011d6b320 x28: ffff0000f66a1d00
[   26.688730] x27: ffff0000f588ecf8 x26: ffff0000f66a1d00
[   26.689202] x25: ffff8000119cf3a8 x24: 0000000001000001
[   26.689675] x23: 0000000080000085 x22: ffff8000100101f8
[   26.690148] x21: ffff800011d6b4d0 x20: ffff0000f66a1d00
[   26.690620] x19: ffff800011d6b390 x18: 0000000000000000
[   26.691092] x17: 0000000000000000 x16: 0000000000000000
[   26.691563] x15: 0000000000000000 x14: 0000000000000137
[   26.692036] x13: 0000000000000355 x12: 000000000000035d
[   26.692507] x11: 0000000000000001 x10: 0000000000000a20
[   26.692979] x9 : ffff800011d6b460 x8 : ffff0000f66a2780
[   26.693450] x7 : 0000000000000001 x6 : ffff800011d6b378
[   26.693922] x5 : 00000000d5300000 x4 : ffff800011806118
[   26.694394] x3 : 0000000092800000 x2 : 0000000000000002
[   26.694866] x1 : ffff0000f66a1d00 x0 : 0000000080000085
[   26.695338] Call trace:
[   26.695563]  do_undefinstr+0x2ec/0x310
[   26.695903]  el1_sync_handler+0x88/0x110
[   26.696255]  el1_sync+0x7c/0x100
[   26.696548]  unwind_frame+0x78/0x1a0
[   26.696869]  return_address+0x58/0x90
[   26.697200]  preempt_count_add+0xb8/0x158
[   26.697564]  _raw_spin_lock_irqsave+0x28/0xa0
[   26.697954]  prepare_to_wait_event+0x24/0xe8
[   26.698339]  rk3x_i2c_xfer+0x2c8/0x448
[   26.698675]  __i2c_transfer+0x144/0x650
[   26.699018]  i2c_transfer+0x60/0x128
[   26.699338]  i2c_transfer_buffer_flags+0x5c/0x88
[   26.699752]  regmap_i2c_write+0x20/0x58
[   26.700099]  _regmap_raw_write_impl+0x6f8/0x8c0
[   26.700504]  _regmap_bus_raw_write+0x68/0x88
[   26.700886]  _regmap_write+0x6c/0x160
[   26.701216]  _regmap_update_bits+0xf8/0x110
[   26.701591]  regmap_update_bits_base+0x64/0x98
[   26.701990]  regulator_set_voltage_sel_regmap+0x4c/0x98
[   26.702458]  _regulator_call_set_voltage_sel+0x78/0xd0
[   26.702916]  _regulator_do_set_voltage+0x474/0x5f0
[   26.703343]  regulator_set_voltage_rdev+0xac/0x230
[   26.703771]  regulator_do_balance_voltage+0x280/0x420
[   26.704223]  regulator_balance_voltage+0x50/0x90
[   26.704634]  regulator_set_voltage_unlocked+0x94/0x118
[   26.705090]  regulator_set_voltage+0x50/0x90
[   26.705474]  _set_opp_voltage+0x44/0x110
[   26.705826]  dev_pm_opp_set_rate+0x250/0x618
[   26.706213]  set_target+0x40/0x88 [cpufreq_dt]
[   26.706611]  __cpufreq_driver_target+0x2c4/0x668
[   26.707023]  od_dbs_update+0xbc/0x1a0
[   26.707353]  dbs_work_handler+0x40/0x80
[   26.707700]  process_one_work+0x1c4/0x470
[   26.708060]  worker_thread+0x4c/0x420
[   26.708391]  kthread+0x118/0x150
[   26.708682]  ret_from_fork+0x10/0x34
[   26.709006] Code: f9401bf7 17ffff7d a9025bf5 f9001bf7 (d4210000)
[   26.709549] ---[ end trace 96ce5fbef0a4e2d9 ]---
[   26.709961] note: kworker/5:1[45] exited with preempt_count 2
[   26.710629] ------------[ cut here ]------------
[   26.711057] WARNING: CPU: 5 PID: 0 at kernel/rcu/tree.c:618 rcu_eqs_enter.isr                        a.0+0x150/0x168
[   26.711802] Modules linked in: r8152 snd_soc_hdmi_codec hantro_vpu(C) rockchi                        p_vdec(C) snd_soc_rockchip_i2s rockchipdrm v4l2_h264 rockchip_rga videobuf2_dma_                        contig dw_mipi_dsi snd_soc_core zstd videobuf2_vmalloc videobuf2_dma_sg v4l2_mem                        2mem dw_hdmi videobuf2_memops snd_pcm_dmaengine analogix_dp leds_pwm snd_pcm vid                        eobuf2_v4l2 drm_kms_helper panfrost videobuf2_common snd_timer cec gpio_charger                         videodev gpu_sched rc_core snd pwm_fan sg soundcore fusb30x(C) mc drm drm_panel_                        orientation_quirks gpio_beeper cpufreq_dt zram lm75 ip_tables x_tables autofs4 r                        aid10 raid1 raid0 multipath linear raid456 async_raid6_recov async_memcpy async_                        pq async_xor async_tx realtek md_mod dwmac_rk stmmac_platform stmmac mdio_xpcs a                        dc_keys
[   26.717407] CPU: 5 PID: 0 Comm: swapper/5 Tainted: G      D  C        5.8.16-                        rockchip64 #20.08.13
[   26.718188] Hardware name: Helios64 (DT)
[   26.718541] pstate: 200003c5 (nzCv DAIF -PAN -UAO BTYPE=--)
[   26.719037] pc : rcu_eqs_enter.isra.0+0x150/0x168
[   26.719457] lr : rcu_eqs_enter.isra.0+0x1c/0x168
[   26.719867] sp : ffff800011c5bf00
[   26.720164] x29: ffff800011c5bf00 x28: 0000000000000000
[   26.720636] x27: ffff0000f6ea6580 x26: 0000000000000000
[   26.721109] x25: 0000000000000000 x24: ffff80001125f1f8
[   26.721581] x23: ffff0000f6ea6580 x22: ffff8000114f2738
[   26.722053] x21: ffff8000117fa2a0 x20: ffff8000117f9980
[   26.722525] x19: ffff8000114f47c0 x18: 000000000000000e
[   26.722997] x17: 0000000000000001 x16: 0000000000000019
[   26.723468] x15: 0000000000000004 x14: 0000000000000149
[   26.723940] x13: 0000000000000003 x12: 0000000000000000
[   26.724412] x11: 0000000000000040 x10: ffff800011819518
[   26.724884] x9 : ffff800011819510 x8 : ffff0000f6800028
[   26.725356] x7 : 0000000000000000 x6 : 000000003be2522a
[   26.725832] x5 : 00ffffffffffffff x4 : 0000000000002774
[   26.726304] x3 : ffff8000114de018 x2 : 4000000000000000
[   26.726776] x1 : 4000000000000002 x0 : ffff0000f77c87c0
[   26.727248] Call trace:
[   26.727472]  rcu_eqs_enter.isra.0+0x150/0x168
[   26.727864]  rcu_idle_enter+0x10/0x20
[   26.728193]  do_idle+0x20c/0x288
[   26.728484]  cpu_startup_entry+0x28/0x68
[   26.728837]  secondary_start_kernel+0x140/0x178
[   26.729240] ---[ end trace 96ce5fbef0a4e2da ]---
[   36.086089] EXT4-fs (mmcblk0p1): resized to 3670016 blocks
 

 

Strange, that kind of crash should be fixed since 20.08.8. I don't encounter such crash on latest image. Could you get the serial console log start from power up?

[raoulh]

On 10/19/2020 at 8:47 AM, gprovost said:

 

We will do an announcement soon but we realized we made a design mistake on the Helios64 2.5GbE port (LAN2 | eth1) which makes it not perform well in 1000Mb/s mode, however no impact in 2500baseT mode.

 

Regarding your iperf test, seems a bit funny. You sure your routing table is correct for traffic to go really to the expected interface since you are on the same subnet.

Can you do the same tests, but for each test disconnect the cable of the interface you are not testing.... or at least use different subnet.

Also can you check interface speed with ethtool (ethtool eth0 and ethtool eth1).

 

Actually you were right about the routing table, all trafic goes through the 2.5G interface. If I test the 1G interface alone I get correct results:

 

➜  ~ iperf3 -c 192.168.0.87 -p 12345
Connecting to host 192.168.0.87, port 12345
[  5] local 192.168.0.238 port 35700 connected to 192.168.0.87 port 12345
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   112 MBytes   938 Mbits/sec   57    160 KBytes       
[  5]   1.00-2.00   sec   112 MBytes   942 Mbits/sec   45    214 KBytes       
[  5]   2.00-3.00   sec   112 MBytes   938 Mbits/sec   37    206 KBytes       
[  5]   3.00-4.00   sec   112 MBytes   943 Mbits/sec   33    144 KBytes       
[  5]   4.00-5.00   sec   111 MBytes   935 Mbits/sec   53    185 KBytes       
[  5]   5.00-6.00   sec   112 MBytes   940 Mbits/sec   56    160 KBytes       
[  5]   6.00-7.00   sec   112 MBytes   941 Mbits/sec   56    202 KBytes       
[  5]   7.00-8.00   sec   112 MBytes   939 Mbits/sec   43    192 KBytes       
[  5]   8.00-9.00   sec   112 MBytes   939 Mbits/sec   38    201 KBytes       
[  5]   9.00-10.00  sec   112 MBytes   940 Mbits/sec   44    158 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  1.09 GBytes   940 Mbits/sec  462             sender
[  5]   0.00-10.00  sec  1.09 GBytes   938 Mbits/sec                  receiver

 

But with the 2.5G only connected to the 10G switch this is what I get:

 

➜  ~ iperf3 -c 192.168.0.86 -p 12345                
Connecting to host 192.168.0.86, port 12345
[  5] local 192.168.0.238 port 42958 connected to 192.168.0.86 port 12345
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  64.5 MBytes   541 Mbits/sec  509   18.4 KBytes       
[  5]   1.00-2.00   sec  64.1 MBytes   538 Mbits/sec  531   17.0 KBytes       
[  5]   2.00-3.00   sec  68.4 MBytes   574 Mbits/sec  599   18.4 KBytes       
[  5]   3.00-4.00   sec  63.6 MBytes   533 Mbits/sec  488   19.8 KBytes       
[  5]   4.00-5.00   sec  62.3 MBytes   522 Mbits/sec  461   18.4 KBytes       
[  5]   5.00-6.00   sec  63.2 MBytes   530 Mbits/sec  528   26.9 KBytes       
[  5]   6.00-7.00   sec  71.8 MBytes   602 Mbits/sec  622   25.5 KBytes       
[  5]   7.00-8.00   sec  62.8 MBytes   527 Mbits/sec  541   19.8 KBytes       
[  5]   8.00-9.00   sec  62.5 MBytes   524 Mbits/sec  472   18.4 KBytes       
[  5]   9.00-10.00  sec  69.3 MBytes   582 Mbits/sec  588   19.8 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   653 MBytes   547 Mbits/sec  5339             sender
[  5]   0.00-10.00  sec   652 MBytes   547 Mbits/sec                  receiver

 

And when I connect eth1 to a 1G switch this is even worse:

➜  ~ iperf3 -c 192.168.0.86 -p 12345
Connecting to host 192.168.0.86, port 12345
[  5] local 192.168.0.238 port 39408 connected to 192.168.0.86 port 12345
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  15.9 MBytes   134 Mbits/sec  330   25.5 KBytes       
[  5]   1.00-2.00   sec  18.3 MBytes   153 Mbits/sec  436   19.8 KBytes       
[  5]   2.00-3.00   sec  18.2 MBytes   153 Mbits/sec  428   15.6 KBytes       
[  5]   3.00-4.00   sec  9.69 MBytes  81.3 Mbits/sec  277   2.83 KBytes       
[  5]   4.00-5.00   sec  3.54 MBytes  29.7 Mbits/sec   58   2.83 KBytes       
[  5]   5.00-6.00   sec  13.7 MBytes   115 Mbits/sec  270   2.83 KBytes       
[  5]   6.00-7.00   sec  12.9 MBytes   108 Mbits/sec  349   14.1 KBytes       
[  5]   7.00-8.00   sec  12.3 MBytes   103 Mbits/sec  352   4.24 KBytes       
[  5]   8.00-9.00   sec  12.7 MBytes   106 Mbits/sec  307   4.24 KBytes       
[  5]   9.00-10.00  sec  6.15 MBytes  51.6 Mbits/sec  142   11.3 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   123 MBytes   103 Mbits/sec  2949             sender
[  5]   0.00-10.20  sec   123 MBytes   101 Mbits/sec                  receiver

 

Something is actually wrong with the 2.5G port...

 

Edit: forgot the ethtool when connected on the 10G switch:

 

╭─root@helios64 ~  
╰─➤  ethtool eth1      
Settings for eth1:
        Supported ports: [ MII ]
        Supported link modes:   10baseT/Half 10baseT/Full 
                                100baseT/Half 100baseT/Full 
                                1000baseT/Full 
                                2500baseT/Full 
        Supported pause frame use: No
        Supports auto-negotiation: Yes
        Supported FEC modes: Not reported
        Advertised link modes:  10baseT/Half 10baseT/Full 
                                100baseT/Half 100baseT/Full 
                                1000baseT/Full 
                                2500baseT/Full 
        Advertised pause frame use: Symmetric Receive-only
        Advertised auto-negotiation: Yes
        Advertised FEC modes: Not reported
        Link partner advertised link modes:  10baseT/Half 10baseT/Full 
                                             100baseT/Half 100baseT/Full 
                                             1000baseT/Half 1000baseT/Full 
        Link partner advertised pause frame use: No
        Link partner advertised auto-negotiation: Yes
        Link partner advertised FEC modes: Not reported
        Speed: 2500Mb/s
        Duplex: Full
        Port: MII
        PHYAD: 32
        Transceiver: internal
        Auto-negotiation: on
        Supports Wake-on: pumbg
        Wake-on: g
        Current message level: 0x00007fff (32767)
                               drv probe link timer ifdown ifup rx_err tx_err tx_queued intr tx_done rx_status pktdata hw wol
        Link detected: yes

 

Edited October 20, 2020 by raoulh

[gprovost]

10 minutes ago, raoulh said:

Something is actually wrong with the 2.5G port...

 

Yes as I mentioned previously there is design issue with the 2.5GbE interface when used in 1000Mb/s mode (What you see in your last iperf), however no impact in 2500baseT mode.

So now why you get only 547 Mbits/s when connected to your 10G switch it's strange... maybe your 10G port doesn't support 2500baseT mode.

As requested in my last message can show the output of ethtool eth1 on Helios64 when connected to your 10G port switch.

 

Edit: ok so it's in 2500Mb/s speed mode. How's you machine running iperf server is connected to the switch ? 1G or 10G or ?

[raoulh]

5 minutes ago, gprovost said:

Edit: ok so it's in 2500Mb/s speed mode. How's you machine running iperf server is connected to the switch ? 1G or 10G or ?

 

My machine is connected with a 10G SFP+ and is working great with other 10G devices I have on my network.

[flower]

11 minutes ago, raoulh said:

 

My machine is connected with a 10G SFP+ and is working great with other 10G devices I have on my network.

What todes ethtool eth1 show?

root@ghost:~# ethtool eth1
Settings for eth1:
        Supported ports: [ MII ]
        Supported link modes:   10baseT/Half 10baseT/Full 
                                100baseT/Half 100baseT/Full 
                                1000baseT/Full 
                                2500baseT/Full 
        Supported pause frame use: No
        Supports auto-negotiation: Yes
        Supported FEC modes: Not reported
        Advertised link modes:  10baseT/Half 10baseT/Full 
                                100baseT/Half 100baseT/Full 
                                1000baseT/Full 
                                2500baseT/Full 
        Advertised pause frame use: Symmetric Receive-only
        Advertised auto-negotiation: Yes
        Advertised FEC modes: Not reported
        Link partner advertised link modes:  100baseT/Half 100baseT/Full 
                                             1000baseT/Full 
        Link partner advertised pause frame use: No
        Link partner advertised auto-negotiation: Yes
        Link partner advertised FEC modes: Not reported
        Speed: 2500Mb/s
        Duplex: Full
        Port: MII
        PHYAD: 32
        Transceiver: internal
        Auto-negotiation: on
        Supports Wake-on: pumbg
        Wake-on: g
        Current message level: 0x00007fff (32767)
                               drv probe link timer ifdown ifup rx_err tx_err tx_queued intr tx_done rx_status pktdata hw wol
        Link detected: yes

 

the important part is: "Speed: 2500MB/s"

my iperf3 shows 1.8GBit/s

 

it is possible that your switch doesnt support 2.5Gbe. Could you post vendor and model?

 

[gprovost]

14 minutes ago, raoulh said:

My machine is connected with a 10G SFP+ and is working great with other 10G devices I have on my network.

 

What do you get when you use -R option on iperf3 client side in order do a download test from your PC to Helios64.

 

Here my iperf3 just done now.

 

PC (iperf3 -s) <-- 5G --> Netgear MS510TX <-- 2.5G -->  Helios64 (iperf3 -c)

 

root@helios64:~# iperf3 -c 10.10.20.254
Connecting to host 10.10.20.254, port 5201
[  5] local 10.10.20.3 port 56716 connected to 10.10.20.254 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   155 MBytes  1.30 Gbits/sec    0   2.13 MBytes       
[  5]   1.00-2.00   sec   168 MBytes  1.41 Gbits/sec    0   2.26 MBytes       
[  5]   2.00-3.00   sec   166 MBytes  1.39 Gbits/sec    0   5.03 MBytes       
[  5]   3.00-4.00   sec   166 MBytes  1.39 Gbits/sec    0   5.03 MBytes       
[  5]   4.00-5.00   sec   168 MBytes  1.41 Gbits/sec    0   6.23 MBytes       
[  5]   5.00-6.00   sec   168 MBytes  1.41 Gbits/sec    0   6.23 MBytes       
[  5]   6.00-7.00   sec   168 MBytes  1.41 Gbits/sec    0   6.23 MBytes       
[  5]   7.00-8.00   sec   168 MBytes  1.40 Gbits/sec    0   6.23 MBytes       
[  5]   8.00-9.00   sec   166 MBytes  1.39 Gbits/sec    0   6.23 MBytes       
[  5]   9.00-10.00  sec   169 MBytes  1.42 Gbits/sec    0   6.23 MBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  1.62 GBytes  1.39 Gbits/sec    0             sender
[  5]   0.00-10.00  sec  1.62 GBytes  1.39 Gbits/sec                  receiver

iperf Done.


root@helios64:~# iperf3 -c 10.10.20.254 -R
Connecting to host 10.10.20.254, port 5201
Reverse mode, remote host 10.10.20.254 is sending
[  5] local 10.10.20.3 port 56720 connected to 10.10.20.254 port 5201
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec   268 MBytes  2.24 Gbits/sec                  
[  5]   1.00-2.00   sec   280 MBytes  2.35 Gbits/sec                  
[  5]   2.00-3.00   sec   272 MBytes  2.28 Gbits/sec                  
[  5]   3.00-4.00   sec   280 MBytes  2.35 Gbits/sec                  
[  5]   4.00-5.00   sec   280 MBytes  2.35 Gbits/sec                  
[  5]   5.00-6.00   sec   280 MBytes  2.35 Gbits/sec                  
[  5]   6.00-7.00   sec   280 MBytes  2.35 Gbits/sec                  
[  5]   7.00-8.00   sec   278 MBytes  2.33 Gbits/sec                  
[  5]   8.00-9.00   sec   278 MBytes  2.33 Gbits/sec                  
[  5]   9.00-10.00  sec   280 MBytes  2.34 Gbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  2.71 GBytes  2.33 Gbits/sec  702             sender
[  5]   0.00-10.00  sec  2.71 GBytes  2.33 Gbits/sec                  receiver

 

As you can see the sending perf are not good because of the TX offloading feature disable.

 

But the RX perf are as expected.

 

 

[registr123]

On 10/19/2020 at 5:31 AM, aprayoga said:

 

The LED brightness is not adjustable. You would need to use semi transparent plastic/tape to reduce the brightness, or change the resistor on front panel board.

 

 

 

10x for the answer - I was actually trying to turn them off completely, but only the blue status light and keeping them going to red on errors etc. 

[raoulh]

23 minutes ago, gprovost said:

 

What do you get when you use -R option on iperf3 client side in order do a download test from your PC to Helios64.

 

Here my iperf3 just done now.

 

PC (iperf3 -s) <-- 5G --> Netgear MS510TX <-- 2.5G -->  Helios64 (iperf3 -c)

 

As you can see the sending perf are not good because of the TX offloading feature disable.

 

But the RX perf are as expected.

 

 

 

Ok, so I think I found the culprit. The helios64 is plugged in a SFP+ 10GBase-T module in a mikrotik switch. It seems that the SFP module I have does not support well the 10GBase-N modes (2.5 and 5Gb).

I tried connecting the helios64 directly on a machine with an 10G ethernet port (no sfp module involved) and I correctly get the speed.

 

➜  ~ iperf3 -c 192.168.0.86 -p 12345
Connecting to host 192.168.0.86, port 12345
[  5] local 192.168.0.238 port 41362 connected to 192.168.0.86 port 12345
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   268 MBytes  2.25 Gbits/sec    0   1.56 MBytes       
[  5]   1.00-2.00   sec   281 MBytes  2.36 Gbits/sec    0   1.56 MBytes       
[  5]   2.00-3.00   sec   280 MBytes  2.35 Gbits/sec    0   1.56 MBytes       
[  5]   3.00-4.00   sec   280 MBytes  2.35 Gbits/sec    0   1.56 MBytes

 

I need to get another 10GBase-T SFP+ module that correctly support 2.5.

 

Thanks all for help!

[dancgn]

Today, home from doing stuff with my kids, the server won't work. No connection with ssh or serial.

 

After restart the server runs about 3-4 Minutes and then this messages was shown:

 

https://www.directupload.net/file/d/5977/5duv2mqg_png.htm

https://www.directupload.net/file/d/5977/jhn6v7wz_png.htm

 

I completley reinstalled the server, but the problem stayed. Rebooting everytime without doing something. And the startup (Starting kernel ...) needs a lot of time.

 

Help...

 

Greets

Daniel

 

-> My german is much better...

[flower]

1 hour ago, dancgn said:

Today, home from doing stuff with my kids, the server won't work. No connection with ssh or serial.

 

After restart the server runs about 3-4 Minutes and then this messages was shown:

 

https://www.directupload.net/file/d/5977/5duv2mqg_png.htm

https://www.directupload.net/file/d/5977/jhn6v7wz_png.htm

 

I completley reinstalled the server, but the problem stayed. Rebooting everytime without doing something. And the startup (Starting kernel ...) needs a lot of time.

 

Help...

 

Greets

Daniel

 

-> My german is much better...

 

it is easier to read logfiles. in putty under logging you can define where to write it.

 

---

Halloo/

Ich schreibe auch lieber auf Deutsch xD Am besten mal unter putty das Logging einschalten. Mit den Dateien kann man dann mehr anfangen.

[inconsistant]

13 hours ago, aprayoga said:

I forgot to answer this. the power staggering is handled by bootloader and not user configurable.
We have tested various HDD brand/model and the highest current during spin up can go up to 7 seconds. We are considering the delay to be user configurable in future.

 

Strange, that kind of crash should be fixed since 20.08.8. I don't encounter such crash on latest image. Could you get the serial console log start from power up?

 

My logs are actually from a serial console start up.  The logs I captured was a fresh SD card with 20.08.13 on the first time boot.

 

1 hour ago, dancgn said:

Today, home from doing stuff with my kids, the server won't work. No connection with ssh or serial.

 

After restart the server runs about 3-4 Minutes and then this messages was shown:

 

https://www.directupload.net/file/d/5977/5duv2mqg_png.htm

https://www.directupload.net/file/d/5977/jhn6v7wz_png.htm

 

I completley reinstalled the server, but the problem stayed. Rebooting everytime without doing something. And the startup (Starting kernel ...) needs a lot of time.

 

Help...

 

Greets

Daniel

 

-> My german is much better...

 

I had the same problem, you can see my logs above.  I was getting the same errors after the update from 20.08.10 to 20.08.13, but also on first boot on a fresh 20.08.13 as well.

[dancgn]

vor 22 Minuten schrieb inconsistant:

 

My logs are actually from a serial console start up.  The logs I captured was a fresh SD card with 20.08.13 on the first time boot.

 

 

I had the same problem, you can see my logs above.  I was getting the same errors after the update from 20.08.10 to 20.08.13, but also on first boot on a fresh 20.08.13 as well.

 

So wait for a new image, or back to 20.08.10...

 

vor 29 Minuten schrieb flower:

 

it is easier to read logfiles. in putty under logging you can define where to write it.

 

---

Halloo/

Ich schreibe auch lieber auf Deutsch xD Am besten mal unter putty das Logging einschalten. Mit den Dateien kann man dann mehr anfangen.

 

Okay...jemand der Deutsch spricht  inconsistant hat ja wohl das gleiche Problem. Also warten auf Update oder zurück zur alten Version...

[flower]

a short feedback to the latest versions:

.10 - most stable so far. three days without a problem

.11 - crashed once in 12h

.12 - crashed once in 12h

.13 - seems ok so far. running for 12h

 

with .13 i get these errors though (verbosity=6 in /boot/armbianEnv.txt, no rsyslog and systemd-journald redirected to serial):

[   15.801564] systemd-udevd[466]: Process '/bin/ln -sf /sys/devices/platform/ff120000.i2c/i2c-2/2-004c/hwmon/hwmon2 /dev/thermal-board' failed with exit code 1.
[   15.804618] systemd-udevd[491]: Process '/bin/ln -sf /sys/devices/virtual/thermal/thermal_zone0/hwmon0 /dev/thermal-cpu' failed with exit code 1.

 

[aprayoga]

10 hours ago, inconsistant said:

 

My logs are actually from a serial console start up.  The logs I captured was a fresh SD card with 20.08.13 on the first time boot.

You should see something like this when capturing serial console from power on

Spoiler

DDR Version 1.24 20191016
In
soft reset
SRX
channel 0
CS = 0
MR0=0x18
MR4=0x1
MR5=0x1
MR8=0x10
MR12=0x72
MR14=0x72
MR18=0x0
MR19=0x0
MR24=0x8
MR25=0x0
channel 1
CS = 0
MR0=0x18
MR4=0x1
MR5=0x1
MR8=0x10
MR12=0x72
MR14=0x72
MR18=0x0
MR19=0x0
MR24=0x8
MR25=0x0
channel 0 training pass!
channel 1 training pass!
change freq to 416MHz 0,1
Channel 0: LPDDR4,416MHz
Bus Width=32 Col=10 Bank=8 Row=16 CS=1 Die Bus-Width=16 Size=2048MB
Channel 1: LPDDR4,416MHz
Bus Width=32 Col=10 Bank=8 Row=16 CS=1 Die Bus-Width=16 Size=2048MB
256B stride
channel 0
CS = 0
MR0=0x18
MR4=0x1
MR5=0x1
MR8=0x10
MR12=0x72
MR14=0x72
MR18=0x0
MR19=0x0
MR24=0x8
MR25=0x0
channel 1
CS = 0
MR0=0x18
MR4=0x1
MR5=0x1
MR8=0x10
MR12=0x72
MR14=0x72
MR18=0x0
MR19=0x0
MR24=0x8
MR25=0x0
channel 0 training pass!
channel 1 training pass!
channel 0, cs 0, advanced training done
channel 1, cs 0, advanced training done
change freq to 856MHz 1,0
ch 0 ddrconfig = 0x101, ddrsize = 0x40
ch 1 ddrconfig = 0x101, ddrsize = 0x40
pmugrf_os_reg[2] = 0x32C1F2C1, stride = 0xD
ddr_set_rate to 328MHZ
ddr_set_rate to 666MHZ
ddr_set_rate to 928MHZ
channel 0, cs 0, advanced training done
channel 1, cs 0, advanced training done
ddr_set_rate to 416MHZ, ctl_index 0
ddr_set_rate to 856MHZ, ctl_index 1
support 416 856 328 666 928 MHz, current 856MHz
OUT
Boot1: 2019-03-14, version: 1.19
CPUId = 0x0
ChipType = 0x10, 324
SdmmcInit=2 0
BootCapSize=100000
UserCapSize=14910MB
FwPartOffset=2000 , 100000
mmc0:cmd5,20
SdmmcInit=0 0
BootCapSize=0
UserCapSize=15193MB
FwPartOffset=2000 , 0
StorageInit ok = 77398
SecureMode = 0
SecureInit read PBA: 0x4
SecureInit read PBA: 0x404
SecureInit read PBA: 0x804
SecureInit read PBA: 0xc04
SecureInit read PBA: 0x1004
SecureInit read PBA: 0x1404
SecureInit read PBA: 0x1804
SecureInit read PBA: 0x1c04
SecureInit ret = 0, SecureMode = 0
atags_set_bootdev: ret:(0)
GPT 0x3380ec0 signature is wrong
recovery gpt...
GPT 0x3380ec0 signature is wrong
recovery gpt fail!
LoadTrust Addr:0x4000
No find bl30.bin
No find bl32.bin
Load uboot, ReadLba = 2000
Load OK, addr=0x200000, size=0xdd6b0
RunBL31 0x40000
NOTICE:  BL31: v1.3(debug):42583b6
NOTICE:  BL31: Built : 07:55:13, Oct 15 2019
NOTICE:  BL31: Rockchip release version: v1.1
INFO:    GICv3 with legacy support detected. ARM GICV3 driver initialized in EL3
INFO:    Using opteed sec cpu_context!
INFO:    boot cpu mask: 0
INFO:    plat_rockchip_pmu_init(1190): pd status 3e
INFO:    BL31: Initializing runtime services
WARNING: No OPTEE provided by BL2 boot loader, Booting device without OPTEE initialization. SMC`s destined for OPTEE will return SMC_UNK
ERROR:   Error initializing runtime service opteed_fast
INFO:    BL31: Preparing for EL3 exit to normal world
INFO:    Entry point address = 0x200000
INFO:    SPSR = 0x3c9


U-Boot 2020.07-armbian (Oct 19 2020 - 08:25:23 +0200)

SoC: Rockchip rk3399
Reset cause: RST
DRAM:  3.9 GiB
PMIC:  RK808
SF: Detected w25q128 with page size 256 Bytes, erase size 4 KiB, total 16 MiB
MMC:   mmc@fe320000: 1, sdhci@fe330000: 0
Loading Environment from MMC... *** Warning - bad CRC, using default environment

In:    serial
Out:   serial
Err:   serial
Model: Helios64
Revision: 1.2 - 4GB non ECC
Net:   eth0: ethernet@fe300000
scanning bus for devices...
Hit any key to stop autoboot:  0
switch to partitions #0, OK
mmc1 is current device
Scanning mmc 1:1...
Found U-Boot script /boot/boot.scr
3185 bytes read in 5 ms (622.1 KiB/s)
## Executing script at 00500000
Boot script loaded from mmc 1
117 bytes read in 5 ms (22.5 KiB/s)
14090231 bytes read in 600 ms (22.4 MiB/s)
27275776 bytes read in 1158 ms (22.5 MiB/s)
79946 bytes read in 13 ms (5.9 MiB/s)
2698 bytes read in 9 ms (292 KiB/s)
Applying kernel provided DT fixup script (rockchip-fixup.scr)
## Executing script at 09000000
## Loading init Ramdisk from Legacy Image at 06000000 ...
   Image Name:   uInitrd
   Image Type:   AArch64 Linux RAMDisk Image (gzip compressed)
   Data Size:    14090167 Bytes = 13.4 MiB
   Load Address: 00000000
   Entry Point:  00000000
   Verifying Checksum ... OK
## Flattened Device Tree blob at 01f00000
   Booting using the fdt blob at 0x1f00000
   Loading Ramdisk to f5176000, end f5ee5fb7 ... OK
   Loading Device Tree to 00000000f50fa000, end 00000000f5175fff ... OK

Starting kernel ...

 

 

 

We are investigating what change causing this crash, what changes on upstream kernel could have an impact.

Meanwhile, if you want to revert to previous kernel, 5.8.14, used on Armbian 20.08.10, you could use armbian-config > System > Other

 

or you could also get previous image from https://archive.armbian.com/helios64/archive/

 

Edited October 21, 2020 by aprayoga

[jbergler]

If you, like myself, installed on eMMC and are experiencing the crashes on 20.08.14 - I booted up via a 20.08.10 sdcard and fixed the environment on emmc

 

# mount the emmc + get ready to chroot
mkdir /mnt/chroot
mount /dev/mmcblk1p2 /mnt/chroot/
mount /dev/mmcblk1p1 /mnt/chroot/media/mmcboot
mount --bind /mnt/chroot/media/mmcboot/boot/ /mnt/chroot/boot/
mount --bind /dev /mnt/chroot/dev/
mount --bind /proc /mnt/chroot/proc/
mount --bind /tmp /mnt/chroot/tmp/

# chroot in and downgrade to 20.08.10
chroot /mnt/chroot/ /bin/bash
apt install \
  linux-dtb-current-rockchip64=20.08.10 \
  linux-headers-current-rockchip64=20.08.10 \
  linux-image-current-rockchip64=20.08.10 \
  armbian-config=20.08.10 \
  armbian-firmware=20.08.10 \
  linux-focal-root-current-helios64=20.08.10 \
  linux-u-boot-helios64-current=20.08.10

exit
# now remove the sd card and hit reset

 

@aprayoga It's probably unrelated, but while working through the above I noticed that I ran out of space on /boot.

I installed to eMMC the first version that was working, if that helps. I chose f2fs when I installed on eMMAC and this is the resulting partition layout

mmcblk1      179:32   0 14.6G  0 disk 
├─mmcblk1p1  179:33   0   96M  0 part
└─mmcblk1p2  179:34   0 14.3G  0 part
mmcblk1boot0 179:64   0    4M  1 disk 
mmcblk1boot1 179:96   0    4M  1 disk

Sadly I didn't grab enough info from what was in the boot partition before I nuked it and reinstalled the appropriate packages.

 

Edited October 21, 2020 by jbergler
fixed typo

[jbergler]

@aprayoga if you still need it here's a full boot log of the crash (the actual stacktrace of the crash is inconsistent for me)

 

Spoiler

DDR Version 1.24 20191016

In

A: 0x1804

SecureInit read PBA: 0x1c04

SecureInit ret = 0, SecureMode = 0

atags_set_bootdev: ret:(0)

GPT 0x3380ec0 signature is wrong

recovery gpt...

GPT 0x3380ec0 signature is wrong

recovery gpt fail!

LoadTrust Addr:0x4000

No find bl30.bin

No find bl32.bin

Load uboot, ReadLba = 2000

Load OK, addr=0x200000, size=0xdcfd8

RunBL31 0x40000

NOTICE:  BL31: v1.3(debug):42583b6

NOTICE:  BL31: Built : 07:55:13, Oct 15 2019

NOTICE:  BL31: Rockchip release version: v1.1

INFO:    GICv3 with legacy support detected. ARM GICV3 driver initialized in EL3

INFO:    Using opteed sec cpu_context!

INFO:    boot cpu mask: 0

INFO:    plat_rockchip_pmu_init(1190): pd status 3e

INFO:    BL31: Initializing runtime services

WARNING: No OPTEE provided by BL2 boot loader, Booting device without OPTEE initialization. SMC`s destined for OPTEE will return SMC_UNK

ERROR:   Error initializing runtime service opteed_fast

INFO:    BL31: Preparing for EL3 exit to normal world

INFO:    Entry point address = 0x200000

INFO:    SPSR = 0x3c9

 

 

U-Boot 2020.07-armbian (Oct 13 2020 - 16:49:59 +0200)

 

SoC: Rockchip rk3399

Reset cause: POR

DRAM:  3.9 GiB

PMIC:  RK808

SF: Detected w25q128 with page size 256 Bytes, erase size 4 KiB, total 16 MiB

MMC:   mmc@fe320000: 1, sdhci@fe330000: 0

Loading Environment from MMC... scanning bus for devices...

Hit any key to stop autoboot:  0

Card did not respond to voltage select!

switch to partitions #0, OK

mmc0(part 0) is current device

Scanning mmc 0:1...

Found U-Boot script /boot/boot.scr

27236864 bytes read in 2646 ms (9.8 MiB/s)

 

Starting kernel ...

 

[   16.291853] systemd[1]: Failed to start Import ZFS pools by cache file.

[   16.768236] systemd[1]: Failed to start Set console font and keymap.

ckchipdrm analogix_dp dw_hdmi dw_mipi_dsi drm_kms_helper cec rc_core drm adc_keys drm_panel_orientation_quirks

[   23.935692] CPU: 4 PID: 0 Comm: swapper/4 Tainted: G         C        5.8.16-rockchip64 #20.08.14

[   23.936474] Hardware name: Helios64 (DT)

[   23.936827] pstate: 80000085 (Nzcv daIf -PAN -UAO BTYPE=--)

[   23.937336] pc : __update_load_avg_se+0x64/0x308

[   23.937751] lr : update_load_avg+0x278/0x5e0

[   23.938131] sp : ffff800011abbd80

[   23.938428] x29: ffff800011abbd80 x28: 0000000000000001

[   23.938901] x27: ffff0000f6ea5700 x26: ffff800011abc000

[   23.939373] x25: ffff8000114f1d20 x24: 0000000000000000

[   23.939846] x23: ffff0000f77a7b40 x22: 0000000000000005

[   23.940317] x21: 000000058fae126f x20: ffff0000f61c80c0

[   23.940789] x19: ffff0000f61c80c0 x18: 0000000000000000

[   23.941261] x17: 0000000000000000 x16: 0000000000000000

[   23.941733] x15: 0000000000000000 x14: 0000000000000000

[   23.942205] x13: 0000000000000332 x12: 0000000000000339

[   23.942676] x11: 0000000000000001 x10: 0000000000000a20

[   23.943148] x9 : ffff800011c53e70 x8 : 0000000000000000

[   23.943620] x7 : 0000000000000000 x6 : 0000000019d0a216

[   23.944092] x5 : 0000000000000000 x4 : 000000000000005e

[   23.944563] x3 : 0000000000000000 x2 : 000000058fae1000

[   23.945034] x1 : ffff0000f77a7bc0 x0 : 000000000000005e

[   23.945508] Call trace:

[   23.945734]  __update_load_avg_se+0x64/0x308

[   23.946118]  update_load_avg+0x278/0x5e0

[   23.946472]  enqueue_entity+0x68/0x228

[   23.946809]  enqueue_task_fair+0x78/0x4d8

[   23.947171]  ttwu_do_activate+0x84/0xe8

[   23.947516]  sched_ttwu_pending+0xc0/0x168

[   23.947884]  flush_smp_call_function_queue+0xec/0x258

[   23.948334]  generic_smp_call_function_single_interrupt+0x14/0x20

[   23.948879]  handle_IPI+0x258/0x3e8

[   23.949195]  gic_handle_irq+0x154/0x158

[   23.949539]  el1_irq+0xb8/0x180

[   23.949824]  arch_cpu_idle+0x28/0x218

[   23.950158]  default_idle_call+0x1c/0x44

[   23.950510]  do_idle+0x210/0x288

[   23.950801]  cpu_startup_entry+0x24/0x68

[   23.951155]  secondary_start_kernel+0x140/0x178

[   23.951564] Code: 2a0403e0 8b030042 f9402023 f9006262 (b9401cc1)

[   23.952116] ---[ end trace fd422e59e1cb21cd ]---

[   23.952527] Kernel panic - not syncing: Fatal exception in interrupt

[   23.953092] SMP: stopping secondary CPUs

[   23.953448] Kernel Offset: disabled

[   23.953763] CPU features: 0x240022,2000600c

[   23.954135] Memory Limit: none

[   23.954422] ---[ end Kernel panic - not syncing: Fatal exception in interrupt ]---

 

Spoiler

channel 0

CS = 0

MR0=0x18

MR4=0x1

MR5=0x1

MR8=0x10

MR12=0x72

MR14=0x72

MR18=0x0

MR19=0x0

MR24=0x8

MR25=0x0

channel 1

CS = 0

MR0=0x18

MR4=0x1

MR5=0x1

MR8=0x10

MR12=0x72

MR14=0x72

MR18=0x0

MR19=0x0

MR24=0x8

MR25=0x0

channel 0 training pass!

channel 1 training pass!

change freq to 416MHz 0,1

Channel 0: LPDDR4,416MHz

Bus Width=32 Col=10 Bank=8 Row=16 CS=1 Die Bus-Width=16 Size=2048MB

Channel 1: LPDDR4,416MHz

Bus Width=32 Col=10 Bank=8 Row=16 CS=1 Die Bus-Width=16 Size=2048MB

256B stride

channel 0

CS = 0

MR0=0x18

MR4=0x1

MR5=0x1

MR8=0x10

MR12=0x72

MR14=0x72

MR18=0x0

MR19=0x0

MR24=0x8

MR25=0x0

channel 1

CS = 0

MR0=0x18

MR4=0x1

MR5=0x1

MR8=0x10

MR12=0x72

MR14=0x72

MR18=0x0

MR19=0x0

MR24=0x8

MR25=0x0

channel 0 training pass!

channel 1 training pass!

channel 0, cs 0, advanced training done

channel 1, cs 0, advanced training done

change freq to 856MHz 1,0

ch 0 ddrconfig = 0x101, ddrsize = 0x40

ch 1 ddrconfig = 0x101, ddrsize = 0x40

pmugrf_os_reg[2] = 0x32C1F2C1, stride = 0xD

ddr_set_rate to 328MHZ

ddr_set_rate to 666MHZ

ddr_set_rate to 928MHZ

channel 0, cs 0, advanced training done

ll return SMC_UNK

ERROR:   Error initializing runtime service opteed_fast

INFO:    BL31: Preparing for EL3 exit to normal world

INFO:    Entry point address = 0x200000

INFO:    SPSR = 0x3c9

 

 

U-Boot 2020.07-armbian (Oct 13 2020 - 16:49:59 +0200)

 

SoC: Rockchip rk3399

Reset cause: POR

DRAM:  3.9 GiB

PMIC:  RK808

eth0: ethernet@fe300000

scanning bus for devices...

Hit any key to stop autoboot:  0

Card did not respond to voltage select!

Scanning mmc 0:1...

Found U-Boot script /boot/boot.scr

3185 bytes read in 18 ms (171.9 KiB/s)

## Executing script at 00500000

Boot script loaded from mmc 0

166 bytes read in 12 ms (12.7 KiB/s)

16311106 bytes read in 1601 ms (9.7 MiB/s)

27236864 bytes read in 2645 ms (9.8 MiB/s)

OK

## Flattened Device Tree blob at 01f00000

   Booting using the fdt blob at 0x1f00000

   Loading Ramdisk to f4f58000, end f5ee6302 ... OK

   Loading Device Tree to 00000000f4edc000, end 00000000f4f57fff ... OK

[   15.396135] OF: graph: no port node found in /syscon@ff770000/usb2-phy@e450/otg-port

[   16.304499] systemd[1]: Failed to start Import ZFS pools by cache file.

[   16.784363] systemd[1]: Failed to start Set console font and keymap.

[   21.126776] kernel BUG at arch/arm64/kernel/traps.c:470!

[   21.127260] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP

[   21.127750] Modules linked in: rfkill governor_performance zstd zram r8152 snd_soc_hdmi_codec leds_pwm gpio_charger pwm_fan snd_soc_rockchip_i2s snd_soc_core snd_pcm_dmaengine panfrost snd_pcm snd_timer snd soundcore gpu_sched sg rockchip_rga hantro_vpu(C) rockchip_vdec(C) videobuf2_dma_sg v4l2_h264 videobuf2_dma_contig videobuf2_vmalloc v4l2_mem2mem videobuf2_memopsx5 : 00000000d5300000 x4 : ffff800011806118

[   21.142542] x3 : 00000000d6500000 x2 : 0000000000000002

[   21.143014] x1 : ffff0000f6ea6580 x0 : 0000000040000085

[   21.143488] Call trace:

[   21.143717]  do_undefinstr+0x2ec/0x310

[   21.144061]  el1_sync_handler+0x88/0x110

[   21.144414]  el1_sync+0x7c/0x100

[   21.144713]  check_preemption_disabled+0x48/0x108

[   21.145143]  irq_exit+0x70/0xe0

[   21.145434]  handle_IPI+0x25c/0x3e8

[   21.145758]  gic_handle_irq+0x154/0x158

[   21.146103]  el1_irq+0xb8/0x180

[   21.146386]  arch_cpu_idle+0x28/0x218

[   21.146721]  default_idle_call+0x1c/0x44

[   21.147082]  do_idle+0x210/0x288

[   21.147373]  cpu_startup_entry+0x28/0x68

[   21.147725]  secondary_start_kernel+0x140/0x178

[   21.148137] Code: f9401bf7 17ffff7d a9025bf5 f9001bf7 (d4210000)

[   21.148691] ---[ end trace 9f0e999f37a83ba9 ]---

[   21.149105] Kernel panic - not syncing: Attempted to kill the idle task!

[   21.149704] SMP: stopping secondary CPUs

[   21.150064] Kernel Offset: disabled

[   21.150378] CPU features: 0x240022,2000600c

[   21.150750] Memory Limit: none

[   21.151036] ---[ end Kernel panic - not syncing: Attempted to kill the idle task! ]---