Lime2 mainline kernel with Debian 9 (stretch) becomes unresponsive (forced reboot required)

[bananapinas]

After over 20 days uptime without a freeze I got 2 freezes within 24hours. I never rolled back though, running > 20 days with 

linux-image-next-sunxi: 5.41
linux-dtb-next-sunxi: 5.41
linux-u-boot-bananapipro-next: 5.41

 

Rolled these 3 now back to 5.31 - let's see...    should I downgrade the linux-headers also?

[JacksGT]

Does anyone have any idea yet where these freezes are coming from / what causes them?

[inaciose]

Hello, same problem here, on a banana pi m1 with armbian 5.38 kernel 4.14.18-sunxi.

The pi hangs frequently, sometimes not even an hour, and kworker shows an unusual cpu usage.

After reading above posts about using the 4.11.5-sunxi kernel, I made a sdcard with armbian 5.31, and freeze the kernel on 4.11.5-sunxi.

Then install and config a ipsec l2pt vpn that I never get to work with kernel 4.14.18, and then it stay idle for more than usual without hang.

If it hangs on the next few days i log it here.

 

 

[Lars_H]

Hello, I have also a problem with kernel freeze on BananaPro (4.14.18-sunxi #24 SMP Fri Feb 9 16:24:32 CET 2018 armv7l GNU/Linux; Debian 9) after one day.

I found few kernel messages:

dmesg -w

...
[58771.275977] brcmfmac: brcmf_psm_watchdog_notify: PSM's watchdog has fired!
[67404.029617] brcmfmac: brcmf_psm_watchdog_notify: PSM's watchdog has fired!
...

from wifi driver:  brcmfmac

Is this possibly the cause of the kernel freeze?

 

Edited March 22, 2018 by Lars_H

[inaciose]

I have  banana pi m1 to test, one with 4.11.5 kernel other with 4.14.18, boot up at same time

The kernel run for 5 days, 19:00:24, then stop working, the kernel 4.11.5 still runs

 

[Igor]

Going back is one solution, but this way might be better:

 

[bananapinas]

With linux-image-next-sunxi, linux-dtb-next-sunxi and linux-u-boot-bananapipro-next = 5.31: up time 39 days

 

[denox]

On 3/22/2018 at 1:51 PM, Igor said:

Going back is one solution, but this way might be better

Just letting you know that I undid the kernel freeze about two weeks ago and upgraded to the latest stable release.

Since then I've been running fine (for 14 days now) on ARMBIAN 5.38 stable Debian GNU/Linux 9 (stretch) 4.14.18-sunxi.

 

* linux-dtb-next-sunxi 5.41

* linux-headers-next-sunxi 5.41

* linux-image-next-sunxi 5.41 

* linux-stretch-root-next-lime2 5.38

* linux-u-boot-lime2-next 5.371 

Since my last report of this issue in late February, I've done some clean-up on packages, disk space and cron jobs as I often have mariadb v15.1 resulting in an out of memory event on my A20 Lime2 machine (1GB memory) while I run my nightly update script (apt, git, composer, npm, pip). This is even after some fine-tuning and achieving a maximum possible memory usage for mariadb of 351.6M (35.30% of installed RAM).  Annoying, but not a big issue.

 

This clean-up activity may have helped with preventing the unresponsiveness issue from occurring again.  The only other difference is that I'm now on 4.14.18-sunxi instead of 4.14.15-sunxi. Perhaps that's what resolved it? Eitherway, my issue has now been resolved. Thanks!

[cyablo]

Problem also occurs on Debian Jessie. Banana Pi M1 started to freeze a few weeks ago, switiching to nighlty build 5.44 does not help. Did a complete reinstall with Debian Stretch, same Problem, froze in just under 4 hours. No log entrys, nothing  Will try switching to Stretch nightlies to see if that helps.

 

By the way: BPi is used headless as Pi-hole and OSCam Server with USB Card-Reader attached, nothing else. Around 10% CPU-load and below 100kb/s network trafic. I do some monitoring with LibreNMS, but just before it froze, everything looked fine. No spikes on all the graphs.

[cyablo]

Freezed on latest nigthlie after around 10 hours

[thepro]

First of all thank you all for this wonderful project!

 

And thank you for tracking down this issue. I had deployed two banana pi pros and hat always this issues (freeze without logs). I tried all usual troubleshooting (power, sd card, ...) but nothing resolved the issue. The only fix for me was to downgrade (i'm using the mainline debian server version)  as a few post earlier suggested (THANK YOU). Both pis are now running stable for more than 14 days  on old kernel bananapi 4.11.5-sunxi #3 SMP armv7l GNU/Linux. I'm really happy because this made me mad, as they are both remote devices that became unresponsive after a 1-4 days. 

Somehow it strange because I'm running also a third pi with the latest (not nightly)  Linux bananapi 4.14.18-sunxi #24 SMP armv7l GNU/Linux version, this one with attached SATA disk and it is working fine for month. I'm doing a daily reboot (which didn't helped with the other two pis) and the system is under heavy load. The two pis (that do not work stable with the new kernel) do not use the sata connector, one of them uses a usb disk and the other pi is running without any external devices attached.

Maybe this may help on tracking the issue.

 

 

Here are the commands that I've run to downgrade the kernel on for the banana pi pro

 

apt install linux-image-next-sunxi=5.31 linux-dtb-next-sunxi=5.31

apt install linux-u-boot-bananapipro-next=5.31

 

run armbian-config and configure that new kernel updates are ignored for the time being as this issue is resolved

 

 

Edited May 21, 2018 by thepro

[raschid]

2 hours ago, thepro said:

... with attached SATA disk...

 

I have the same issue on a BananaPi M1 with an attached SATA disk.

It is running SAMBA, a media server, an iperf server, it monitors and log weather data from an attached weather station and does some home automation.

 

"Freeze" means the system is becoming unresponsive to ssh, cron etc. while still responding to ping and the the reset button. A cpu-load triggered system LED remains solid on.

 

The system ran completely stable on Bananian for nearly three years until I switched to armbian in January.

Initially the freezes occured quite infrequently with the systmem running ok for weeks.
I installed a daemon to monitor system load and dump "ps aux" is load exceeds 2 - but that never happens:  The system becomes unresponsive before the daemon can kick in.

 

After upgrading last Saturday it froze after just 22 hours. I am travelling right now - it's quite unsettling to think of the little bugger running at max overload for the next 5 days.

 

I will downgrade as soon as I am back as @theprosuggests.

 

TL;DR: Current armbians builds on A20 platforms seem to be unstable and have been for some time. Upgrading the current builds seems to make things worse.

[raschid]

OK, so the BPi M1 is still alive - after 6 days of this full-load zombie state. Two more M1s are currently joining it to test stability:

• BPi M1 #1: Armbian 5.38 - kernel 4.11.5
• BPi M1 #2: same Armbian version - kernel 4.14.18
• BPi M1 #3: Armbian 5.45 - kernel 4.17.0 rc7 (to see if stabilty improves)

I will post results ...

[vlad59]

I have a Banana Pi (The very first one, before the Banana Pro / m1 came out) running Armbian 5.38 with kernel 4.14.18 and it runs 24/7. Its uptime is 56 days (and I rebooted the board) so no problem at all.

 

I have another Banana Pi running Armbian 5.31 with kernel 4.11.6 and its uptime is 143 days. I never had a single problem with both of them.

[raschid]

Thx for the info @vlad59 - I would not recommend upgrading to more current versions at the moment.

[piknew]

Hi, my OPIs after upgrading to mainline also started to be unresponsive. Today OPI+2e - before it died (ping was working but rest of services - not) following was dumped to kernel.log:

Spoiler

 

[11145.875753] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: data=ordered,commit=600,errors=remount-ro
[20634.129971] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: data=ordered,commit=600,errors=remount-ro
[44339.174560] kworker/0:0H: page allocation failure: order:0, mode:0x1080020(GFP_ATOMIC), nodemask=(null)
[44339.174589] kworker/0:0H cpuset=/ mems_allowed=0
[44339.174605] CPU: 0 PID: 12047 Comm: kworker/0:0H Not tainted 4.14.18-sunxi #24
[44339.174608] Hardware name: Allwinner sun8i Family
[44339.174626] Workqueue: xprtiod rpc_async_schedule
[44339.174652] [<c010db15>] (unwind_backtrace) from [<c010a0d9>] (show_stack+0x11/0x14)
[44339.174663] [<c010a0d9>] (show_stack) from [<c0868f69>] (dump_stack+0x69/0x78)
[44339.174676] [<c0868f69>] (dump_stack) from [<c01dddd1>] (warn_alloc+0xa1/0x120)
[44339.174687] [<c01dddd1>] (warn_alloc) from [<c01dea25>] (__alloc_pages_nodemask+0xb6d/0xc2c)
[44339.174695] [<c01dea25>] (__alloc_pages_nodemask) from [<c01dec0f>] (page_frag_alloc+0xd3/0xdc)
[44339.174706] [<c01dec0f>] (page_frag_alloc) from [<c0772453>] (__netdev_alloc_skb+0x77/0xbc)
[44339.174722] [<c0772453>] (__netdev_alloc_skb) from [<c06a5cd3>] (stmmac_poll+0x597/0x6e4)
[44339.174736] [<c06a5cd3>] (stmmac_poll) from [<c0782ced>] (net_rx_action+0x115/0x2f0)
[44339.174746] [<c0782ced>] (net_rx_action) from [<c01014a1>] (__do_softirq+0xb9/0x25c)
[44339.174758] [<c01014a1>] (__do_softirq) from [<c011ef27>] (irq_exit+0x7f/0xc4)
[44339.174773] [<c011ef27>] (irq_exit) from [<c0159e4d>] (__handle_domain_irq+0x49/0x84)
[44339.174783] [<c0159e4d>] (__handle_domain_irq) from [<c01013b5>] (gic_handle_irq+0x39/0x68)
[44339.174791] [<c01013b5>] (gic_handle_irq) from [<c010aa25>] (__irq_svc+0x65/0x94)
[44339.174795] Exception stack(0xc1613a30 to 0xc1613a78)
[44339.174802] 3a20:                                     2e9ef000 0000b1ac 0000cc08 0000ab9d
[44339.174809] 3a40: c0dd8b00 c1613c74 c1613ad4 ef012000 0000001c 00000008 c0dd9600 c0d03f48
[44339.174815] 3a60: 60090193 c1613a80 c01e91eb c01e9294 20090133 ffffffff
[44339.174826] [<c010aa25>] (__irq_svc) from [<c01e9294>] (shrink_inactive_list+0x1b0/0x440)
[44339.174836] [<c01e9294>] (shrink_inactive_list) from [<c01e9a45>] (shrink_node_memcg+0x249/0x4b0)
[44339.174844] [<c01e9a45>] (shrink_node_memcg) from [<c01e9d4d>] (shrink_node+0xa1/0x1fc)
[44339.174852] [<c01e9d4d>] (shrink_node) from [<c01e9f69>] (do_try_to_free_pages+0xc1/0x270)
[44339.174859] [<c01e9f69>] (do_try_to_free_pages) from [<c01ea215>] (try_to_free_pages+0xfd/0x320)
[44339.174868] [<c01ea215>] (try_to_free_pages) from [<c01de323>] (__alloc_pages_nodemask+0x46b/0xc2c)
[44339.174878] [<c01de323>] (__alloc_pages_nodemask) from [<c0216139>] (new_slab+0x201/0x444)
[44339.174887] [<c0216139>] (new_slab) from [<c02175ed>] (___slab_alloc.constprop.14+0x16d/0x1c0)
[44339.174895] [<c02175ed>] (___slab_alloc.constprop.14) from [<c02177b9>] (__kmalloc+0x179/0x1b4)
[44339.174903] [<c02177b9>] (__kmalloc) from [<c084cd63>] (rpc_malloc+0x3f/0x70)
[44339.174914] [<c084cd63>] (rpc_malloc) from [<c08449b1>] (call_allocate+0x51/0xc4)
[44339.174922] [<c08449b1>] (call_allocate) from [<c084d227>] (__rpc_execute+0x6b/0x2ec)
[44339.174933] [<c084d227>] (__rpc_execute) from [<c012d451>] (process_one_work+0x155/0x37c)
[44339.174942] [<c012d451>] (process_one_work) from [<c012dff3>] (worker_thread+0xff/0x404)
[44339.174952] [<c012dff3>] (worker_thread) from [<c0131a91>] (kthread+0xfd/0x104)
[44339.174965] [<c0131a91>] (kthread) from [<c0106719>] (ret_from_fork+0x11/0x38)
[44339.174970] Mem-Info:
[44339.174985] active_anon:4916 inactive_anon:5782 isolated_anon:0
 active_file:72639 inactive_file:154029 isolated_file:28
 unevictable:0 dirty:0 writeback:14573 unstable:146
 slab_reclaimable:6772 slab_unreclaimable:4999
 mapped:4809 shmem:2852 pagetables:324 bounce:0
 free:1202 free_pcp:482 free_cma:45
[44339.174994] Node 0 active_anon:19664kB inactive_anon:23128kB active_file:290556kB inactive_file:616116kB unevictable:0kB isolated(anon):0kB isolated(file):112kB mapped:19236kB dirty:0kB writeback:58292kB shmem:11408kB writeback_tmp:0kB unstable:584kB all_unreclaimable? no
[44339.175007] Normal free:4048kB min:3476kB low:4344kB high:5212kB active_anon:6728kB inactive_anon:5156kB active_file:217180kB inactive_file:470540kB unevictable:0kB writepending:44328kB present:786432kB managed:761920kB mlocked:0kB kernel_stack:1040kB pagetables:592kB bounce:0kB free_pcp:1228kB local_pcp:360kB free_cma:0kB
[44339.175009] lowmem_reserve[]: 0 1535 1535
[44339.175025] HighMem free:760kB min:188kB low:412kB high:636kB active_anon:12936kB inactive_anon:17912kB active_file:73376kB inactive_file:145860kB unevictable:0kB writepending:13964kB present:262144kB managed:262132kB mlocked:0kB kernel_stack:0kB pagetables:704kB bounce:0kB free_pcp:700kB local_pcp:216kB free_cma:180kB
[44339.175027] lowmem_reserve[]: 0 0 0
[44339.175034] Normal: 0*4kB 2*8kB (H) 1*16kB (H) 1*32kB (H) 0*64kB 1*128kB (H) 1*256kB (H) 1*512kB (H) 1*1024kB (H) 1*2048kB (H) 0*4096kB = 4032kB
[44339.175068] HighMem: 68*4kB (UC) 51*8kB (UMC) 3*16kB (U) 1*32kB (U) 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 760kB
[44339.175097] 229538 total pagecache pages
[44339.175101] 0 pages in swap cache
[44339.175104] Swap cache stats: add 0, delete 0, find 0/0
[44339.175106] Free swap  = 131068kB
[44339.175108] Total swap = 131068kB
[44339.175110] 262144 pages RAM
[44339.175112] 65536 pages HighMem/MovableOnly
[44339.175114] 6131 pages reserved
[44339.175116] 16384 pages cma reserved
[44339.175188] dwmac-sun8i 1c30000.ethernet: packet dropped
[55271.062033] kworker/0:0H: page allocation failure: order:0, mode:0x1080020(GFP_ATOMIC), nodemask=(null)
[55271.062702] kworker/0:0H cpuset=/ mems_allowed=0
[55271.063225] CPU: 0 PID: 21267 Comm: kworker/0:0H Not tainted 4.14.18-sunxi #24
[55271.063347] Hardware name: Allwinner sun8i Family
[55271.063793] Workqueue: xprtiod xs_tcp_data_receive_workfn
[55271.064500] [<c010db15>] (unwind_backtrace) from [<c010a0d9>] (show_stack+0x11/0x14)
[55271.064924] [<c010a0d9>] (show_stack) from [<c0868f69>] (dump_stack+0x69/0x78)
[55271.065365] [<c0868f69>] (dump_stack) from [<c01dddd1>] (warn_alloc+0xa1/0x120)
[55271.065797] [<c01dddd1>] (warn_alloc) from [<c01dea25>] (__alloc_pages_nodemask+0xb6d/0xc2c)
[55271.066194] [<c01dea25>] (__alloc_pages_nodemask) from [<c01dec0f>] (page_frag_alloc+0xd3/0xdc)
[55271.066654] [<c01dec0f>] (page_frag_alloc) from [<c0772453>] (__netdev_alloc_skb+0x77/0xbc)
[55271.067153] [<c0772453>] (__netdev_alloc_skb) from [<c06a5a49>] (stmmac_poll+0x30d/0x6e4)
[55271.067604] [<c06a5a49>] (stmmac_poll) from [<c0782ced>] (net_rx_action+0x115/0x2f0)
[55271.068000] [<c0782ced>] (net_rx_action) from [<c01014a1>] (__do_softirq+0xb9/0x25c)
[55271.068468] [<c01014a1>] (__do_softirq) from [<c011ed93>] (do_softirq.part.4+0x33/0x38)
[55271.068950] [<c011ed93>] (do_softirq.part.4) from [<c011ee27>] (__local_bh_enable_ip+0x8f/0x94)
[55271.069398] [<c011ee27>] (__local_bh_enable_ip) from [<c07ba8bb>] (ip_finish_output2+0xff/0x320)
[55271.069804] [<c07ba8bb>] (ip_finish_output2) from [<c07bc4c5>] (ip_output+0xa5/0xf4)
[55271.070204] [<c07bc4c5>] (ip_output) from [<c07cf2cf>] (tcp_transmit_skb+0x3d7/0x788)
[55271.070600] [<c07cf2cf>] (tcp_transmit_skb) from [<c07cf7fd>] (tcp_write_xmit+0x17d/0xb68)
[55271.070979] [<c07cf7fd>] (tcp_write_xmit) from [<c07d0213>] (__tcp_push_pending_frames+0x2b/0x78)
[55271.071457] [<c07d0213>] (__tcp_push_pending_frames) from [<c07cca31>] (tcp_rcv_established+0x459/0x4d0)
[55271.071918] [<c07cca31>] (tcp_rcv_established) from [<c07d41ef>] (tcp_v4_do_rcv+0xab/0x13c)
[55271.072331] [<c07d41ef>] (tcp_v4_do_rcv) from [<c076d7fd>] (__release_sock+0x61/0xa4)
[55271.072737] [<c076d7fd>] (__release_sock) from [<c076d861>] (release_sock+0x21/0x54)
[55271.073177] [<c076d861>] (release_sock) from [<c0848d67>] (xs_tcp_data_receive_workfn+0x5f/0x120)
[55271.073664] [<c0848d67>] (xs_tcp_data_receive_workfn) from [<c012d451>] (process_one_work+0x155/0x37c)
[55271.074092] [<c012d451>] (process_one_work) from [<c012dff3>] (worker_thread+0xff/0x404)
[55271.074538] [<c012dff3>] (worker_thread) from [<c0131a91>] (kthread+0xfd/0x104)
[55271.075017] [<c0131a91>] (kthread) from [<c0106719>] (ret_from_fork+0x11/0x38)
[55271.075166] Mem-Info:
[55271.075767] active_anon:5254 inactive_anon:6349 isolated_anon:0
 active_file:54401 inactive_file:171314 isolated_file:32
 unevictable:0 dirty:18 writeback:13173 unstable:64
 slab_reclaimable:6861 slab_unreclaimable:4605
 mapped:4393 shmem:2864 pagetables:345 bounce:0
 free:1289 free_pcp:645 free_cma:35
[55271.076223] Node 0 active_anon:21016kB inactive_anon:25396kB active_file:217604kB inactive_file:685256kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:17572kB dirty:72kB writeback:52692kB shmem:11456kB writeback_tmp:0kB unstable:256kB all_unreclaimable? no
[55271.076841] Normal free:4312kB min:3476kB low:4344kB high:5212kB active_anon:7576kB inactive_anon:7712kB active_file:165480kB inactive_file:518564kB unevictable:0kB writepending:39200kB present:786432kB managed:761920kB mlocked:0kB kernel_stack:1064kB pagetables:652kB bounce:0kB free_pcp:1732kB local_pcp:40kB free_cma:0kB
[55271.076922] lowmem_reserve[]: 0 1535 1535
[55271.077799] HighMem free:844kB min:188kB low:412kB high:636kB active_anon:13440kB inactive_anon:17684kB active_file:52124kB inactive_file:166692kB unevictable:0kB writepending:13564kB present:262144kB managed:262132kB mlocked:0kB kernel_stack:0kB pagetables:728kB bounce:0kB free_pcp:800kB local_pcp:328kB free_cma:140kB
[55271.077871] lowmem_reserve[]: 0 0 0
[55271.078230] Normal: 31*4kB (UM) 0*8kB 0*16kB 0*32kB 1*64kB (H) 1*128kB (H) 1*256kB (H) 1*512kB (H) 1*1024kB (H) 1*2048kB (H) 0*4096kB = 4156kB
[55271.079922] HighMem: 77*4kB (UMC) 45*8kB (UMC) 9*16kB (UM) 1*32kB (M) 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 844kB
[55271.081407] 228543 total pagecache pages
[55271.081538] 0 pages in swap cache
[55271.081691] Swap cache stats: add 0, delete 0, find 0/0
[55271.081797] Free swap  = 131068kB
[55271.081895] Total swap = 131068kB
[55271.081995] 262144 pages RAM
[55271.082096] 65536 pages HighMem/MovableOnly
[55271.082190] 6131 pages reserved
[55271.082289] 16384 pages cma reserved
[55271.084057] dwmac-sun8i 1c30000.ethernet: fail to alloc skb entry 88
[65638.251697] kworker/0:2H: page allocation failure: order:0, mode:0x1080020(GFP_ATOMIC), nodemask=(null)
[65638.252368] kworker/0:2H cpuset=/ mems_allowed=0
[65638.252881] CPU: 0 PID: 25695 Comm: kworker/0:2H Not tainted 4.14.18-sunxi #24
[65638.253006] Hardware name: Allwinner sun8i Family
[65638.253440] Workqueue: xprtiod xs_tcp_data_receive_workfn
[65638.254129] [<c010db15>] (unwind_backtrace) from [<c010a0d9>] (show_stack+0x11/0x14)
[65638.254552] [<c010a0d9>] (show_stack) from [<c0868f69>] (dump_stack+0x69/0x78)
[65638.254992] [<c0868f69>] (dump_stack) from [<c01dddd1>] (warn_alloc+0xa1/0x120)
[65638.255420] [<c01dddd1>] (warn_alloc) from [<c01dea25>] (__alloc_pages_nodemask+0xb6d/0xc2c)
[65638.255816] [<c01dea25>] (__alloc_pages_nodemask) from [<c01dec0f>] (page_frag_alloc+0xd3/0xdc)
[65638.256275] [<c01dec0f>] (page_frag_alloc) from [<c0772453>] (__netdev_alloc_skb+0x77/0xbc)
[65638.256770] [<c0772453>] (__netdev_alloc_skb) from [<c06a5cd3>] (stmmac_poll+0x597/0x6e4)
[65638.257216] [<c06a5cd3>] (stmmac_poll) from [<c0782ced>] (net_rx_action+0x115/0x2f0)
[65638.257610] [<c0782ced>] (net_rx_action) from [<c01014a1>] (__do_softirq+0xb9/0x25c)
[65638.258051] [<c01014a1>] (__do_softirq) from [<c011ef27>] (irq_exit+0x7f/0xc4)
[65638.258557] [<c011ef27>] (irq_exit) from [<c0159e4d>] (__handle_domain_irq+0x49/0x84)
[65638.258989] [<c0159e4d>] (__handle_domain_irq) from [<c01013b5>] (gic_handle_irq+0x39/0x68)
[65638.259343] [<c01013b5>] (gic_handle_irq) from [<c010aa25>] (__irq_svc+0x65/0x94)
[65638.259514] Exception stack(0xc31e5d90 to 0xc31e5dd8)
[65638.259796] 5d80:                                     c2a841c0 000002c0 c0dd09c4 003e7040
[65638.260156] 5da0: c2a84fc0 00000b50 00005ab8 c2a841c0 ed8f9900 d73df60c 00000000 000005a8
[65638.260456] 5dc0: 00000000 c31e5de0 c07c4537 c07cfc36 a0010033 ffffffff
[65638.260859] [<c010aa25>] (__irq_svc) from [<c07cfc36>] (tcp_write_xmit+0x5b6/0xb68)
[65638.261258] [<c07cfc36>] (tcp_write_xmit) from [<c07d0213>] (__tcp_push_pending_frames+0x2b/0x78)
[65638.261737] [<c07d0213>] (__tcp_push_pending_frames) from [<c07cc805>] (tcp_rcv_established+0x22d/0x4d0)
[65638.262192] [<c07cc805>] (tcp_rcv_established) from [<c07d41ef>] (tcp_v4_do_rcv+0xab/0x13c)
[65638.262608] [<c07d41ef>] (tcp_v4_do_rcv) from [<c076d7fd>] (__release_sock+0x61/0xa4)
[65638.263019] [<c076d7fd>] (__release_sock) from [<c076d861>] (release_sock+0x21/0x54)
[65638.263459] [<c076d861>] (release_sock) from [<c0848d67>] (xs_tcp_data_receive_workfn+0x5f/0x120)
[65638.263939] [<c0848d67>] (xs_tcp_data_receive_workfn) from [<c012d451>] (process_one_work+0x155/0x37c)
[65638.264365] [<c012d451>] (process_one_work) from [<c012dff3>] (worker_thread+0xff/0x404)
[65638.264809] [<c012dff3>] (worker_thread) from [<c0131a91>] (kthread+0xfd/0x104)
[65638.265277] [<c0131a91>] (kthread) from [<c0106719>] (ret_from_fork+0x11/0x38)
[65638.265417] Mem-Info:
[65638.266011] active_anon:552 inactive_anon:9046 isolated_anon:0
 active_file:26575 inactive_file:199685 isolated_file:0
 unevictable:0 dirty:1 writeback:16905 unstable:370
 slab_reclaimable:8095 slab_unreclaimable:5018
 mapped:1898 shmem:2848 pagetables:285 bounce:0
 free:1331 free_pcp:588 free_cma:125
[65638.266457] Node 0 active_anon:2208kB inactive_anon:36184kB active_file:106300kB inactive_file:798740kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:7592kB dirty:4kB writeback:67620kB shmem:11392kB writeback_tmp:0kB unstable:1480kB all_unreclaimable? no
[65638.267058] Normal free:4088kB min:3476kB low:4344kB high:5212kB active_anon:884kB inactive_anon:8752kB active_file:83476kB inactive_file:601032kB unevictable:0kB writepending:50184kB present:786432kB managed:761920kB mlocked:0kB kernel_stack:1008kB pagetables:340kB bounce:0kB free_pcp:1472kB local_pcp:152kB free_cma:0kB
[65638.267136] lowmem_reserve[]: 0 1535 1535
[65638.267980] HighMem free:1236kB min:188kB low:412kB high:636kB active_anon:1324kB inactive_anon:27432kB active_file:22824kB inactive_file:197680kB unevictable:0kB writepending:17404kB present:262144kB managed:262132kB mlocked:0kB kernel_stack:0kB pagetables:800kB bounce:0kB free_pcp:880kB local_pcp:108kB free_cma:500kB
[65638.268052] lowmem_reserve[]: 0 0 0
[65638.268411] Normal: 0*4kB 1*8kB (H) 1*16kB (H) 1*32kB (H) 1*64kB (H) 1*128kB (H) 1*256kB (H) 1*512kB (H) 1*1024kB (H) 1*2048kB (H) 0*4096kB = 4088kB
[65638.270242] HighMem: 135*4kB (UMC) 69*8kB (UMC) 5*16kB (U) 2*32kB (U) 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1236kB
[65638.271731] 229122 total pagecache pages
[65638.271863] 0 pages in swap cache
[65638.272017] Swap cache stats: add 0, delete 0, find 0/0
[65638.272121] Free swap  = 131068kB
[65638.272218] Total swap = 131068kB
[65638.272318] 262144 pages RAM
[65638.272419] 65536 pages HighMem/MovableOnly
[65638.272513] 6131 pages reserved
[65638.272609] 16384 pages cma reserved
[65638.274204] dwmac-sun8i 1c30000.ethernet: packet dropped
[65638.278124] ksoftirqd/0: page allocation failure: order:0, mode:0x1080020(GFP_ATOMIC), nodemask=(null)
[65638.278967] ksoftirqd/0 cpuset=/ mems_allowed=0
[65638.279570] CPU: 0 PID: 7 Comm: ksoftirqd/0 Not tainted 4.14.18-sunxi #24
[65638.279737] Hardware name: Allwinner sun8i Family
[65638.280387] [<c010db15>] (unwind_backtrace) from [<c010a0d9>] (show_stack+0x11/0x14)
[65638.280817] [<c010a0d9>] (show_stack) from [<c0868f69>] (dump_stack+0x69/0x78)
[65638.281250] [<c0868f69>] (dump_stack) from [<c01dddd1>] (warn_alloc+0xa1/0x120)
[65638.281670] [<c01dddd1>] (warn_alloc) from [<c01dea25>] (__alloc_pages_nodemask+0xb6d/0xc2c)
[65638.282074] [<c01dea25>] (__alloc_pages_nodemask) from [<c01dec0f>] (page_frag_alloc+0xd3/0xdc)
[65638.282527] [<c01dec0f>] (page_frag_alloc) from [<c0772453>] (__netdev_alloc_skb+0x77/0xbc)
[65638.283018] [<c0772453>] (__netdev_alloc_skb) from [<c06a5cd3>] (stmmac_poll+0x597/0x6e4)
[65638.283451] [<c06a5cd3>] (stmmac_poll) from [<c0782ced>] (net_rx_action+0x115/0x2f0)
[65638.283850] [<c0782ced>] (net_rx_action) from [<c01014a1>] (__do_softirq+0xb9/0x25c)
[65638.284280] [<c01014a1>] (__do_softirq) from [<c011ef27>] (irq_exit+0x7f/0xc4)
[65638.284873] [<c011ef27>] (irq_exit) from [<c0159e4d>] (__handle_domain_irq+0x49/0x84)
[65638.285430] [<c0159e4d>] (__handle_domain_irq) from [<c01013b5>] (gic_handle_irq+0x39/0x68)
[65638.285799] [<c01013b5>] (gic_handle_irq) from [<c010aa25>] (__irq_svc+0x65/0x94)
[65638.285970] Exception stack(0xef119e58 to 0xef119ea0)
[65638.286189] 9e40:                                                       ee1e9b00 00000000
[65638.286557] 9e60: 00000000 0000774e c0cb71c0 ee04fc00 ee1e9b00 ef119eb8 ef6a61c0 2e9ef000
[65638.286925] 9e80: ef0fd100 c08773fd ef11803c ef119ea8 c08773fd c01377a6 00010033 ffffffff
[65638.287325] [<c010aa25>] (__irq_svc) from [<c01377a6>] (finish_task_switch+0x72/0x18c)
[65638.287914] [<c01377a6>] (finish_task_switch) from [<c08773fd>] (__schedule+0x185/0x6c8)
[65638.288415] [<c08773fd>] (__schedule) from [<c087796f>] (schedule+0x2f/0x68)
[65638.288908] [<c087796f>] (schedule) from [<c01348ff>] (smpboot_thread_fn+0x12b/0x130)
[65638.289403] [<c01348ff>] (smpboot_thread_fn) from [<c0131a91>] (kthread+0xfd/0x104)
[65638.289873] [<c0131a91>] (kthread) from [<c0106719>] (ret_from_fork+0x11/0x38)
[65638.291897] dwmac-sun8i 1c30000.ethernet: packet dropped
[65638.297504] kworker/0:2H: page allocation failure: order:0, mode:0x1080020(GFP_ATOMIC), nodemask=(null)
[65638.298072] kworker/0:2H cpuset=/ mems_allowed=0
[65638.298559] CPU: 0 PID: 25695 Comm: kworker/0:2H Not tainted 4.14.18-sunxi #24
[65638.298685] Hardware name: Allwinner sun8i Family
[65638.299065] Workqueue: xprtiod xs_tcp_data_receive_workfn
[65638.299682] [<c010db15>] (unwind_backtrace) from [<c010a0d9>] (show_stack+0x11/0x14)
[65638.300080] [<c010a0d9>] (show_stack) from [<c0868f69>] (dump_stack+0x69/0x78)
[65638.300487] [<c0868f69>] (dump_stack) from [<c01dddd1>] (warn_alloc+0xa1/0x120)
[65638.300909] [<c01dddd1>] (warn_alloc) from [<c01dea25>] (__alloc_pages_nodemask+0xb6d/0xc2c)
[65638.301309] [<c01dea25>] (__alloc_pages_nodemask) from [<c01dec0f>] (page_frag_alloc+0xd3/0xdc)
[65638.301761] [<c01dec0f>] (page_frag_alloc) from [<c0772453>] (__netdev_alloc_skb+0x77/0xbc)
[65638.302245] [<c0772453>] (__netdev_alloc_skb) from [<c06a5cd3>] (stmmac_poll+0x597/0x6e4)
[65638.302677] [<c06a5cd3>] (stmmac_poll) from [<c0782ced>] (net_rx_action+0x115/0x2f0)
[65638.303058] [<c0782ced>] (net_rx_action) from [<c01014a1>] (__do_softirq+0xb9/0x25c)
[65638.303483] [<c01014a1>] (__do_softirq) from [<c011ef27>] (irq_exit+0x7f/0xc4)
[65638.304159] [<c011ef27>] (irq_exit) from [<c0159e4d>] (__handle_domain_irq+0x49/0x84)
[65638.304592] [<c0159e4d>] (__handle_domain_irq) from [<c01013b5>] (gic_handle_irq+0x39/0x68)
[65638.304951] [<c01013b5>] (gic_handle_irq) from [<c010aa25>] (__irq_svc+0x65/0x94)
[65638.305285] Exception stack(0xc31e5cd0 to 0xc31e5d18)
[65638.305557] 5cc0:                                     9fce0000 00004b73 00000000 00000000
[65638.305925] 5ce0: 003bddeb ed8f9900 0000096e 00000000 73f9c000 0000096e eda8a000 d73e0cac
[65638.306232] 5d00: 10000000 c31e5d20 c07c6567 c08671d4 00010033 ffffffff
[65638.306603] [<c010aa25>] (__irq_svc) from [<c08671d4>] (__do_div64+0x58/0xc6)
[65638.307034] [<c08671d4>] (__do_div64) from [<c07c6567>] (tcp_update_pacing_rate+0x9f/0xa4)
[65638.307528] [<c07c6567>] (tcp_update_pacing_rate) from [<c07cb513>] (tcp_ack+0xc0b/0xd78)
[65638.308186] [<c07cb513>] (tcp_ack) from [<c07cc74b>] (tcp_rcv_established+0x173/0x4d0)
[65638.308647] [<c07cc74b>] (tcp_rcv_established) from [<c07d41ef>] (tcp_v4_do_rcv+0xab/0x13c)
[65638.309059] [<c07d41ef>] (tcp_v4_do_rcv) from [<c076d7fd>] (__release_sock+0x61/0xa4)
[65638.309472] [<c076d7fd>] (__release_sock) from [<c076d861>] (release_sock+0x21/0x54)
[65638.310051] [<c076d861>] (release_sock) from [<c0848d67>] (xs_tcp_data_receive_workfn+0x5f/0x120)
[65638.310530] [<c0848d67>] (xs_tcp_data_receive_workfn) from [<c012d451>] (process_one_work+0x155/0x37c)
[65638.310953] [<c012d451>] (process_one_work) from [<c012dff3>] (worker_thread+0xff/0x404)
[65638.311393] [<c012dff3>] (worker_thread) from [<c0131a91>] (kthread+0xfd/0x104)
[65638.311995] [<c0131a91>] (kthread) from [<c0106719>] (ret_from_fork+0x11/0x38)
[65638.313233] dwmac-sun8i 1c30000.ethernet: packet dropped
[65638.317911] kworker/0:2H: page allocation failure: order:0, mode:0x1080020(GFP_ATOMIC), nodemask=(null)
[65638.318438] kworker/0:2H cpuset=/ mems_allowed=0
[65638.319043] CPU: 0 PID: 25695 Comm: kworker/0:2H Not tainted 4.14.18-sunxi #24
[65638.319169] Hardware name: Allwinner sun8i Family
[65638.319603] Workqueue: xprtiod xs_tcp_data_receive_workfn
[65638.320228] [<c010db15>] (unwind_backtrace) from [<c010a0d9>] (show_stack+0x11/0x14)
[65638.320748] [<c010a0d9>] (show_stack) from [<c0868f69>] (dump_stack+0x69/0x78)
[65638.321159] [<c0868f69>] (dump_stack) from [<c01dddd1>] (warn_alloc+0xa1/0x120)
[65638.321755] [<c01dddd1>] (warn_alloc) from [<c01dea25>] (__alloc_pages_nodemask+0xb6d/0xc2c)
[65638.322152] [<c01dea25>] (__alloc_pages_nodemask) from [<c01dec0f>] (page_frag_alloc+0xd3/0xdc)
[65638.322602] [<c01dec0f>] (page_frag_alloc) from [<c0772453>] (__netdev_alloc_skb+0x77/0xbc)
[65638.323072] [<c0772453>] (__netdev_alloc_skb) from [<c06a5cd3>] (stmmac_poll+0x597/0x6e4)
[65638.323599] [<c06a5cd3>] (stmmac_poll) from [<c0782ced>] (net_rx_action+0x115/0x2f0)
[65638.324046] [<c0782ced>] (net_rx_action) from [<c01014a1>] (__do_softirq+0xb9/0x25c)
[65638.324666] [<c01014a1>] (__do_softirq) from [<c011ef27>] (irq_exit+0x7f/0xc4)
[65638.325147] [<c011ef27>] (irq_exit) from [<c0159e4d>] (__handle_domain_irq+0x49/0x84)
[65638.325577] [<c0159e4d>] (__handle_domain_irq) from [<c01013b5>] (gic_handle_irq+0x39/0x68)
[65638.325932] [<c01013b5>] (gic_handle_irq) from [<c010aa25>] (__irq_svc+0x65/0x94)
[65638.326101] Exception stack(0xc31e5ca8 to 0xc31e5cf0)
[65638.326576] 5ca0:                   000005dc 000000fc 000005a8 0000ffff 00000014 000005dc
[65638.326944] 5cc0: c2a855c0 ee4a3940 ed8f9900 000005dc 128e4ad4 00000007 c31e5d38 c31e5cf8
[65638.327183] 5ce0: c077015d c0770174 80010033 ffffffff
[65638.327617] [<c010aa25>] (__irq_svc) from [<c0770174>] (skb_gso_validate_mtu+0x2c/0x68)
[65638.328227] [<c0770174>] (skb_gso_validate_mtu) from [<c07bbc59>] (ip_finish_output+0x111/0x194)
[65638.328634] [<c07bbc59>] (ip_finish_output) from [<c07bc4c5>] (ip_output+0xa5/0xf4)
[65638.329023] [<c07bc4c5>] (ip_output) from [<c07cf2cf>] (tcp_transmit_skb+0x3d7/0x788)
[65638.329411] [<c07cf2cf>] (tcp_transmit_skb) from [<c07cf7fd>] (tcp_write_xmit+0x17d/0xb68)
[65638.329949] [<c07cf7fd>] (tcp_write_xmit) from [<c07d0213>] (__tcp_push_pending_frames+0x2b/0x78)
[65638.330597] [<c07d0213>] (__tcp_push_pending_frames) from [<c07cc805>] (tcp_rcv_established+0x22d/0x4d0)
[65638.331047] [<c07cc805>] (tcp_rcv_established) from [<c07d41ef>] (tcp_v4_do_rcv+0xab/0x13c)
[65638.331601] [<c07d41ef>] (tcp_v4_do_rcv) from [<c076d7fd>] (__release_sock+0x61/0xa4)
[65638.331997] [<c076d7fd>] (__release_sock) from [<c076d861>] (release_sock+0x21/0x54)
[65638.332441] [<c076d861>] (release_sock) from [<c0848d67>] (xs_tcp_data_receive_workfn+0x5f/0x120)
[65638.332922] [<c0848d67>] (xs_tcp_data_receive_workfn) from [<c012d451>] (process_one_work+0x155/0x37c)
[65638.333334] [<c012d451>] (process_one_work) from [<c012dff3>] (worker_thread+0xff/0x404)
[65638.333936] [<c012dff3>] (worker_thread) from [<c0131a91>] (kthread+0xfd/0x104)
[65638.334391] [<c0131a91>] (kthread) from [<c0106719>] (ret_from_fork+0x11/0x38)
[65638.335427] dwmac-sun8i 1c30000.ethernet: packet dropped
[65638.338108] kworker/0:2H: page allocation failure: order:0, mode:0x1080020(GFP_ATOMIC), nodemask=(null)
[65638.338775] kworker/0:2H cpuset=/ mems_allowed=0
[65638.339386] CPU: 0 PID: 25695 Comm: kworker/0:2H Not tainted 4.14.18-sunxi #24
[65638.339509] Hardware name: Allwinner sun8i Family
[65638.339832] Workqueue: xprtiod xs_tcp_data_receive_workfn
[65638.340375] [<c010db15>] (unwind_backtrace) from [<c010a0d9>] (show_stack+0x11/0x14)
[65638.340766] [<c010a0d9>] (show_stack) from [<c0868f69>] (dump_stack+0x69/0x78)
[65638.341331] [<c0868f69>] (dump_stack) from [<c01dddd1>] (warn_alloc+0xa1/0x120)
[65638.341750] [<c01dddd1>] (warn_alloc) from [<c01dea25>] (__alloc_pages_nodemask+0xb6d/0xc2c)
[65638.342145] [<c01dea25>] (__alloc_pages_nodemask) from [<c01dec0f>] (page_frag_alloc+0xd3/0xdc)
[65638.342584] [<c01dec0f>] (page_frag_alloc) from [<c0772453>] (__netdev_alloc_skb+0x77/0xbc)
[65638.343048] [<c0772453>] (__netdev_alloc_skb) from [<c06a5cd3>] (stmmac_poll+0x597/0x6e4)
[65638.343464] [<c06a5cd3>] (stmmac_poll) from [<c0782ced>] (net_rx_action+0x115/0x2f0)
[65638.343840] [<c0782ced>] (net_rx_action) from [<c01014a1>] (__do_softirq+0xb9/0x25c)
[65638.344434] [<c01014a1>] (__do_softirq) from [<c011ed93>] (do_softirq.part.4+0x33/0x38)
[65638.344900] [<c011ed93>] (do_softirq.part.4) from [<c011ee27>] (__local_bh_enable_ip+0x8f/0x94)
[65638.345332] [<c011ee27>] (__local_bh_enable_ip) from [<c07ba8bb>] (ip_finish_output2+0xff/0x320)
[65638.345718] [<c07ba8bb>] (ip_finish_output2) from [<c07bc4c5>] (ip_output+0xa5/0xf4)
[65638.346260] [<c07bc4c5>] (ip_output) from [<c07cf2cf>] (tcp_transmit_skb+0x3d7/0x788)
[65638.346645] [<c07cf2cf>] (tcp_transmit_skb) from [<c07cf7fd>] (tcp_write_xmit+0x17d/0xb68)
[65638.347028] [<c07cf7fd>] (tcp_write_xmit) from [<c07d0213>] (__tcp_push_pending_frames+0x2b/0x78)
[65638.347661] [<c07d0213>] (__tcp_push_pending_frames) from [<c07cc805>] (tcp_rcv_established+0x22d/0x4d0)
[65638.348261] [<c07cc805>] (tcp_rcv_established) from [<c07d41ef>] (tcp_v4_do_rcv+0xab/0x13c)
[65638.348811] [<c07d41ef>] (tcp_v4_do_rcv) from [<c076d7fd>] (__release_sock+0x61/0xa4)
[65638.349250] [<c076d7fd>] (__release_sock) from [<c076d861>] (release_sock+0x21/0x54)
[65638.349690] [<c076d861>] (release_sock) from [<c0848d67>] (xs_tcp_data_receive_workfn+0x5f/0x120)
[65638.350342] [<c0848d67>] (xs_tcp_data_receive_workfn) from [<c012d451>] (process_one_work+0x155/0x37c)
[65638.350768] [<c012d451>] (process_one_work) from [<c012dff3>] (worker_thread+0xff/0x404)
[65638.351195] [<c012dff3>] (worker_thread) from [<c0131a91>] (kthread+0xfd/0x104)
[65638.351653] [<c0131a91>] (kthread) from [<c0106719>] (ret_from_fork+0x11/0x38)
[65638.353178] dwmac-sun8i 1c30000.ethernet: packet dropped
[65638.355069] kworker/0:2H: page allocation failure: order:0, mode:0x1080020(GFP_ATOMIC), nodemask=(null)
[65638.355554] kworker/0:2H cpuset=/ mems_allowed=0
[65638.356005] CPU: 0 PID: 25695 Comm: kworker/0:2H Not tainted 4.14.18-sunxi #24
[65638.356463] Hardware name: Allwinner sun8i Family
[65638.356797] Workqueue: xprtiod xs_tcp_data_receive_workfn
[65638.357334] [<c010db15>] (unwind_backtrace) from [<c010a0d9>] (show_stack+0x11/0x14)
[65638.357729] [<c010a0d9>] (show_stack) from [<c0868f69>] (dump_stack+0x69/0x78)
[65638.358271] [<c0868f69>] (dump_stack) from [<c01dddd1>] (warn_alloc+0xa1/0x120)
[65638.358683] [<c01dddd1>] (warn_alloc) from [<c01dea25>] (__alloc_pages_nodemask+0xb6d/0xc2c)
[65638.359078] [<c01dea25>] (__alloc_pages_nodemask) from [<c01dec0f>] (page_frag_alloc+0xd3/0xdc)
[65638.359633] [<c01dec0f>] (page_frag_alloc) from [<c0772453>] (__netdev_alloc_skb+0x77/0xbc)
[65638.360330] [<c0772453>] (__netdev_alloc_skb) from [<c06a5cd3>] (stmmac_poll+0x597/0x6e4)
[65638.360746] [<c06a5cd3>] (stmmac_poll) from [<c0782ced>] (net_rx_action+0x115/0x2f0)
[65638.361122] [<c0782ced>] (net_rx_action) from [<c01014a1>] (__do_softirq+0xb9/0x25c)
[65638.361553] [<c01014a1>] (__do_softirq) from [<c011ed93>] (do_softirq.part.4+0x33/0x38)
[65638.362179] [<c011ed93>] (do_softirq.part.4) from [<c011ee27>] (__local_bh_enable_ip+0x8f/0x94)
[65638.362613] [<c011ee27>] (__local_bh_enable_ip) from [<c07bc4a9>] (ip_output+0x89/0xf4)
[65638.363006] [<c07bc4a9>] (ip_output) from [<c07cf2cf>] (tcp_transmit_skb+0x3d7/0x788)
[65638.363391] [<c07cf2cf>] (tcp_transmit_skb) from [<c07cf7fd>] (tcp_write_xmit+0x17d/0xb68)
[65638.363773] [<c07cf7fd>] (tcp_write_xmit) from [<c07d0213>] (__tcp_push_pending_frames+0x2b/0x78)
[65638.364227] [<c07d0213>] (__tcp_push_pending_frames) from [<c07cc805>] (tcp_rcv_established+0x22d/0x4d0)
[65638.364774] [<c07cc805>] (tcp_rcv_established) from [<c07d41ef>] (tcp_v4_do_rcv+0xab/0x13c)
[65638.365257] [<c07d41ef>] (tcp_v4_do_rcv) from [<c076d7fd>] (__release_sock+0x61/0xa4)
[65638.365862] [<c076d7fd>] (__release_sock) from [<c076d861>] (release_sock+0x21/0x54)
[65638.366309] [<c076d861>] (release_sock) from [<c0848d67>] (xs_tcp_data_receive_workfn+0x5f/0x120)
[65638.366769] [<c0848d67>] (xs_tcp_data_receive_workfn) from [<c012d451>] (process_one_work+0x155/0x37c)
[65638.367204] [<c012d451>] (process_one_work) from [<c012dff3>] (worker_thread+0xff/0x404)
[65638.367640] [<c012dff3>] (worker_thread) from [<c0131a91>] (kthread+0xfd/0x104)
[65638.368096] [<c0131a91>] (kthread) from [<c0106719>] (ret_from_fork+0x11/0x38)
[65638.368989] dwmac-sun8i 1c30000.ethernet: packet dropped
[65638.375152] kworker/0:2H: page allocation failure: order:0, mode:0x1080020(GFP_ATOMIC), nodemask=(null)
[65638.375795] kworker/0:2H cpuset=/ mems_allowed=0
[65638.376253] CPU: 0 PID: 25695 Comm: kworker/0:2H Not tainted 4.14.18-sunxi #24
[65638.376375] Hardware name: Allwinner sun8i Family
[65638.376715] Workqueue: xprtiod xs_tcp_data_receive_workfn
[65638.377272] [<c010db15>] (unwind_backtrace) from [<c010a0d9>] (show_stack+0x11/0x14)
[65638.377667] [<c010a0d9>] (show_stack) from [<c0868f69>] (dump_stack+0x69/0x78)
[65638.378070] [<c0868f69>] (dump_stack) from [<c01dddd1>] (warn_alloc+0xa1/0x120)
[65638.378488] [<c01dddd1>] (warn_alloc) from [<c01dea25>] (__alloc_pages_nodemask+0xb6d/0xc2c)
[65638.378876] [<c01dea25>] (__alloc_pages_nodemask) from [<c01dec0f>] (page_frag_alloc+0xd3/0xdc)
[65638.379325] [<c01dec0f>] (page_frag_alloc) from [<c0772453>] (__netdev_alloc_skb+0x77/0xbc)
[65638.379798] [<c0772453>] (__netdev_alloc_skb) from [<c06a5cd3>] (stmmac_poll+0x597/0x6e4)
[65638.380219] [<c06a5cd3>] (stmmac_poll) from [<c0782ced>] (net_rx_action+0x115/0x2f0)
[65638.380596] [<c0782ced>] (net_rx_action) from [<c01014a1>] (__do_softirq+0xb9/0x25c)
[65638.381027] [<c01014a1>] (__do_softirq) from [<c011ed93>] (do_softirq.part.4+0x33/0x38)
[65638.381493] [<c011ed93>] (do_softirq.part.4) from [<c011ee27>] (__local_bh_enable_ip+0x8f/0x94)
[65638.381972] [<c011ee27>] (__local_bh_enable_ip) from [<c07c6db7>] (tcp_check_space+0x8b/0xd8)
[65638.382466] [<c07c6db7>] (tcp_check_space) from [<c07cc80b>] (tcp_rcv_established+0x233/0x4d0)
[65638.382927] [<c07cc80b>] (tcp_rcv_established) from [<c07d41ef>] (tcp_v4_do_rcv+0xab/0x13c)
[65638.383528] [<c07d41ef>] (tcp_v4_do_rcv) from [<c076d7fd>] (__release_sock+0x61/0xa4)
[65638.383931] [<c076d7fd>] (__release_sock) from [<c076d861>] (release_sock+0x21/0x54)
[65638.384376] [<c076d861>] (release_sock) from [<c0848d67>] (xs_tcp_data_receive_workfn+0x5f/0x120)
[65638.384974] [<c0848d67>] (xs_tcp_data_receive_workfn) from [<c012d451>] (process_one_work+0x155/0x37c)
[65638.385407] [<c012d451>] (process_one_work) from [<c012dff3>] (worker_thread+0xff/0x404)
[65638.385841] [<c012dff3>] (worker_thread) from [<c0131a91>] (kthread+0xfd/0x104)
[65638.386305] [<c0131a91>] (kthread) from [<c0106719>] (ret_from_fork+0x11/0x38)
[65638.387373] dwmac-sun8i 1c30000.ethernet: packet dropped
[65638.388710] kworker/0:2H: page allocation failure: order:0, mode:0x1080020(GFP_ATOMIC), nodemask=(null)
[65638.389191] kworker/0:2H cpuset=/ mems_allowed=0
[65638.389640] CPU: 0 PID: 25695 Comm: kworker/0:2H Not tainted 4.14.18-sunxi #24
[65638.389760] Hardware name: Allwinner sun8i Family
[65638.390063] Workqueue: xprtiod xs_tcp_data_receive_workfn
[65638.390604] [<c010db15>] (unwind_backtrace) from [<c010a0d9>] (show_stack+0x11/0x14)
[65638.390996] [<c010a0d9>] (show_stack) from [<c0868f69>] (dump_stack+0x69/0x78)
[65638.391391] [<c0868f69>] (dump_stack) from [<c01dddd1>] (warn_alloc+0xa1/0x120)
[65638.391981] [<c01dddd1>] (warn_alloc) from [<c01dea25>] (__alloc_pages_nodemask+0xb6d/0xc2c)
[65638.392377] [<c01dea25>] (__alloc_pages_nodemask) from [<c01dec0f>] (page_frag_alloc+0xd3/0xdc)
[65638.392815] [<c01dec0f>] (page_frag_alloc) from [<c0772453>] (__netdev_alloc_skb+0x77/0xbc)
[65638.393286] [<c0772453>] (__netdev_alloc_skb) from [<c06a5cd3>] (stmmac_poll+0x597/0x6e4)
[65638.393701] [<c06a5cd3>] (stmmac_poll) from [<c0782ced>] (net_rx_action+0x115/0x2f0)
[65638.394076] [<c0782ced>] (net_rx_action) from [<c01014a1>] (__do_softirq+0xb9/0x25c)
[65638.394508] [<c01014a1>] (__do_softirq) from [<c011ed93>] (do_softirq.part.4+0x33/0x38)
[65638.394973] [<c011ed93>] (do_softirq.part.4) from [<c011ee27>] (__local_bh_enable_ip+0x8f/0x94)
[65638.395403] [<c011ee27>] (__local_bh_enable_ip) from [<c076d7d5>] (__release_sock+0x39/0xa4)
[65638.395793] [<c076d7d5>] (__release_sock) from [<c076d861>] (release_sock+0x21/0x54)
[65638.396228] [<c076d861>] (release_sock) from [<c0848d67>] (xs_tcp_data_receive_workfn+0x5f/0x120)
[65638.396683] [<c0848d67>] (xs_tcp_data_receive_workfn) from [<c012d451>] (process_one_work+0x155/0x37c)
[65638.397095] [<c012d451>] (process_one_work) from [<c012dff3>] (worker_thread+0xff/0x404)
[65638.397521] [<c012dff3>] (worker_thread) from [<c0131a91>] (kthread+0xfd/0x104)
[65638.397975] [<c0131a91>] (kthread) from [<c0106719>] (ret_from_fork+0x11/0x38)
[65638.398913] dwmac-sun8i 1c30000.ethernet: packet dropped
[65638.402424] kworker/0:2H: page allocation failure: order:0, mode:0x1080020(GFP_ATOMIC), nodemask=(null)
[65638.402922] kworker/0:2H cpuset=/ mems_allowed=0
[65638.403372] CPU: 0 PID: 25695 Comm: kworker/0:2H Not tainted 4.14.18-sunxi #24
[65638.403493] Hardware name: Allwinner sun8i Family
[65638.404009] Workqueue: xprtiod xs_tcp_data_receive_workfn
[65638.404572] [<c010db15>] (unwind_backtrace) from [<c010a0d9>] (show_stack+0x11/0x14)
[65638.404968] [<c010a0d9>] (show_stack) from [<c0868f69>] (dump_stack+0x69/0x78)
[65638.405370] [<c0868f69>] (dump_stack) from [<c01dddd1>] (warn_alloc+0xa1/0x120)
[65638.405776] [<c01dddd1>] (warn_alloc) from [<c01dea25>] (__alloc_pages_nodemask+0xb6d/0xc2c)
[65638.406181] [<c01dea25>] (__alloc_pages_nodemask) from [<c01dec0f>] (page_frag_alloc+0xd3/0xdc)
[65638.406781] [<c01dec0f>] (page_frag_alloc) from [<c0772453>] (__netdev_alloc_skb+0x77/0xbc)
[65638.407249] [<c0772453>] (__netdev_alloc_skb) from [<c06a5cd3>] (stmmac_poll+0x597/0x6e4)
[65638.407662] [<c06a5cd3>] (stmmac_poll) from [<c0782ced>] (net_rx_action+0x115/0x2f0)
[65638.408048] [<c0782ced>] (net_rx_action) from [<c01014a1>] (__do_softirq+0xb9/0x25c)
[65638.408783] [<c01014a1>] (__do_softirq) from [<c011ef27>] (irq_exit+0x7f/0xc4)
[65638.409271] [<c011ef27>] (irq_exit) from [<c0159e4d>] (__handle_domain_irq+0x49/0x84)
[65638.409702] [<c0159e4d>] (__handle_domain_irq) from [<c01013b5>] (gic_handle_irq+0x39/0x68)
[65638.410172] [<c01013b5>] (gic_handle_irq) from [<c010aa25>] (__irq_svc+0x65/0x94)
[65638.410343] Exception stack(0xc31e5cf0 to 0xc31e5d38)
[65638.410614] 5ce0:                                     d73e2884 00000b50 00000000 485176ef
[65638.410971] 5d00: 485176e6 0000000f 00000002 ed8f9900 00000004 c0d03f48 eda8a380 d73e33d4
[65638.411272] 5d20: 00000000 c31e5d40 c07cad93 c07caede 00010033 ffffffff
[65638.412089] [<c010aa25>] (__irq_svc) from [<c07caede>] (tcp_ack+0x5d6/0xd78)
[65638.412582] [<c07caede>] (tcp_ack) from [<c07cca0f>] (tcp_rcv_established+0x437/0x4d0)
[65638.413031] [<c07cca0f>] (tcp_rcv_established) from [<c07d41ef>] (tcp_v4_do_rcv+0xab/0x13c)
[65638.413439] [<c07d41ef>] (tcp_v4_do_rcv) from [<c076d7fd>] (__release_sock+0x61/0xa4)
[65638.413841] [<c076d7fd>] (__release_sock) from [<c076d861>] (release_sock+0x21/0x54)
[65638.414551] [<c076d861>] (release_sock) from [<c0848d67>] (xs_tcp_data_receive_workfn+0x5f/0x120)
[65638.415017] [<c0848d67>] (xs_tcp_data_receive_workfn) from [<c012d451>] (process_one_work+0x155/0x37c)
[65638.415435] [<c012d451>] (process_one_work) from [<c012dff3>] (worker_thread+0xff/0x404)
[65638.415870] [<c012dff3>] (worker_thread) from [<c0131a91>] (kthread+0xfd/0x104)
[65638.416610] [<c0131a91>] (kthread) from [<c0106719>] (ret_from_fork+0x11/0x38)
[65638.417611] dwmac-sun8i 1c30000.ethernet: packet dropped
[65638.421021] kworker/0:2H: page allocation failure: order:0, mode:0x1080020(GFP_ATOMIC), nodemask=(null)
[65638.421648] kworker/0:2H cpuset=/ mems_allowed=0
[65638.422099] CPU: 0 PID: 25695 Comm: kworker/0:2H Not tainted 4.14.18-sunxi #24
[65638.422219] Hardware name: Allwinner sun8i Family
[65638.422549] Workqueue: xprtiod xs_tcp_data_receive_workfn
[65638.423110] [<c010db15>] (unwind_backtrace) from [<c010a0d9>] (show_stack+0x11/0x14)
[65638.423505] [<c010a0d9>] (show_stack) from [<c0868f69>] (dump_stack+0x69/0x78)
[65638.423901] [<c0868f69>] (dump_stack) from [<c01dddd1>] (warn_alloc+0xa1/0x120)
[65638.424316] [<c01dddd1>] (warn_alloc) from [<c01dea25>] (__alloc_pages_nodemask+0xb6d/0xc2c)
[65638.424710] [<c01dea25>] (__alloc_pages_nodemask) from [<c01dec0f>] (page_frag_alloc+0xd3/0xdc)
[65638.425354] [<c01dec0f>] (page_frag_alloc) from [<c0772453>] (__netdev_alloc_skb+0x77/0xbc)
[65638.425816] [<c0772453>] (__netdev_alloc_skb) from [<c06a5cd3>] (stmmac_poll+0x597/0x6e4)
[65638.426235] [<c06a5cd3>] (stmmac_poll) from [<c0782ced>] (net_rx_action+0x115/0x2f0)
[65638.426806] [<c0782ced>] (net_rx_action) from [<c01014a1>] (__do_softirq+0xb9/0x25c)
[65638.427405] [<c01014a1>] (__do_softirq) from [<c011ef27>] (irq_exit+0x7f/0xc4)
[65638.427883] [<c011ef27>] (irq_exit) from [<c0159e4d>] (__handle_domain_irq+0x49/0x84)
[65638.428306] [<c0159e4d>] (__handle_domain_irq) from [<c01013b5>] (gic_handle_irq+0x39/0x68)
[65638.428661] [<c01013b5>] (gic_handle_irq) from [<c010aa25>] (__irq_svc+0x65/0x94)
[65638.428954] Exception stack(0xc31e5dd0 to 0xc31e5e18)
[65638.429225] 5dc0:                                     00000000 c31e4000 2e9ef000 00007761
[65638.429592] 5de0: ef6a61c0 ee1e9b00 ee1e9b00 c31e5e30 c0d03f48 c0cb71c0 ee1e9ea4 c0901eb0
[65638.429892] 5e00: 00000000 c31e5e20 c087734f c0877732 00010033 ffffffff
[65638.430316] [<c010aa25>] (__irq_svc) from [<c0877732>] (__schedule+0x4ba/0x6c8)
[65638.430894] [<c0877732>] (__schedule) from [<c0877b27>] (preempt_schedule_common+0x13/0x20)
[65638.431360] [<c0877b27>] (preempt_schedule_common) from [<c0877b61>] (_cond_resched+0x2d/0x30)
[65638.431801] [<c0877b61>] (_cond_resched) from [<c076d801>] (__release_sock+0x65/0xa4)
[65638.432198] [<c076d801>] (__release_sock) from [<c076d861>] (release_sock+0x21/0x54)
[65638.432781] [<c076d861>] (release_sock) from [<c0848d67>] (xs_tcp_data_receive_workfn+0x5f/0x120)
[65638.433246] [<c0848d67>] (xs_tcp_data_receive_workfn) from [<c012d451>] (process_one_work+0x155/0x37c)
[65638.433665] [<c012d451>] (process_one_work) from [<c012dff3>] (worker_thread+0xff/0x404)
[65638.434098] [<c012dff3>] (worker_thread) from [<c0131a91>] (kthread+0xfd/0x104)
[65638.434553] [<c0131a91>] (kthread) from [<c0106719>] (ret_from_fork+0x11/0x38)
[65638.435417] dwmac-sun8i 1c30000.ethernet: packet dropped

 

[raschid]

 

Hmm, looks like the machine is running out of RAM. Maybe we have some memory leak ...

Maybe you could check for decreasing available memory over some time?

[gounthar]

Thanks to @raschid for giving me the link of this thread, as I may have the same kind of problem with my OrangePi Zero.

Is there anything specific to armbian to monitor the memory usage, or a good old /proc/meminfo is the info to use?
I know my Zero uses swap, which is a bad thing, but it looks like the discussion about using zram on Armbian for small memory SBC is not over...

Here is the diagnosis.

[raschid]

@gounthar

you might try downgrading the kernel:

sudo apt install linux-image-next-sunxi=5.31 linux-dtb-next-sunxi=5.31

sudo apt install linux-u-boot-bananapipro-next=5.31

 

[gounthar]

The first package downgrade went fine.
Unfortunately, the second one does not exist in version 5.31. Which version should I use?

sudo apt install   linux-u-boot-orangepizero-next=5.31
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Version '5.31' for 'linux-u-boot-orangepizero-next' was not found
root@openstf-orangepi-zero:~# sudo apt install   linux-u-boot-orangepizero-next
Reading package lists... Done
Building dependency tree
Reading state information... Done
linux-u-boot-orangepizero-next is already the newest version (5.38).
0 upgraded, 0 newly installed, 0 to remove and 2 not upgraded.

Thanks.

[raschid]

You are right ... and out of luck for now:

https://github.com/armbian/build/issues/1003#issuecomment-394987272

 

you could try a recent nightly build:

https://dl.armbian.com/orangepizero/

 

One BananaPi running a recent kernel (Armbian 5.45 / kernel 4.17.0 rc7) has been up and running for 12 days now ...

[gounthar]

I see. Why not try, thanks a lot @raschid.

[raschid]

Keep us posted. Good luck, @gounthar

[sude]

I have the same problem.

I'm using an original BananaPi as headless server, OMV, and downgraded to 5.31 since it get unresponsive (exact like denox descripes it) after a couple of days with current image installed.

Yesterday, after an uptime of about 10days it got unresponsive again, with the same symptoms as before.

The banana is powered via 5V 3A power adapter over the SATA-Power connector, new SD card, 2,5" HD via SATA.

Would it make sense to install an Armbian Xenial desktop legacy kernel 3.4.y image and give it a try?

[piknew]

For me it did. On all of my boards (Opi+2e, Opi+2, OpiPC, OpiZero) I was trying:

 

• stable "next"
• nightly "next"

Unfortunately it failed (sometimes I was able to get some exceptions in logs, example of one of them is here:

 

 

So I went back to "default" (stable). And on all boards - no issues.

[sude]

My last attempt was switching to nightly, but again the system got unresponsive after a few days.

I'm now installing "Armbian_5.38_Bananapi_Ubuntu_xenial_default_3.4.113_desktop".

Maybe that works stable, if not I think that it's caused by a hardware malfunction...

Unfortunately it seems that in that image it's not possible to install OMV via armbianconfig with all the improvments necessary to run smoothly (shown here).

Sorry, found the menu to install OMV in armbian-config -> softy

after rebooting, the OMV menu item is gone. OK, I stop polluting this thread and start a new one....sorry

[piknew]

Orange Pi Zero (which I am using for "testing"). I have built:

 

Quote

root@PKOTHER:~# uname -a
Linux PKOTHER 4.14.50-sunxi #1 SMP Mon Jun 18 23:35:22 +07 2018 armv7l GNU/Linux

Then I started normal (for me at least) activity) - cron.d is running my own backup software every one hour.

 

This one in general is "rsync" (it is scanning disks/nfs) - determines the differences, storing state in sqlite3 db... So, I guess most of "my use cases".

 

I will let you results. So, far it is OK. I can say that this test "frozen" next kernel (downloaded from stable repository) upgrade after around 1 day...

 

Some results:

 

backup-TEST.run.1.log