zram vs swap

[Stuart Naylor]

I am on a Pi Zero at the moment and no Neon and LZ4 with Zram provides less overhead. I have noticed at extreme short duration LZO can sometime be better but 95% LZ4 is approx 20% lighter than LZO.

I have been reading through and presuming that the binaries for LZO & LZ4 must have better NEON optimisation for the LZO binary as it doesn't make sense that its faster. Also when you have an lowly Arm with no Neon then it pans out that LZ4 is better as you are running both without NEON extensions.

I just wondered if anyone had ever compiled LZ4 with https://github.com/lz4/lz4 with armcc --cpu Cortex-XX --vectorize -O3 -Otime --diag_warning=optimizations source.c ? And maybe given it a go.
I am still trying to find what vm.swappiness you plumed for 100 seems to work best for me as dropping down to 80 raises my overall load by almost 20%

For Pi Zero Zram with vm.swappiness = 90,   /proc/sys/vm/page-cluster = 0 seems to provide lower overall load.

What I am thinking is that ram2log with zram control will run a cron job that will increment / decrement vm.swappiness based on processor load as high vm.swappiness = 90 or above starts to add to the already intense boot. But after the first 2 minutes when things are much quieter its reducing load by almost 20% but there is no dynamic adjustment and it certainly could benefit from it.
Anyone any ideas of maybe something more eloquent than cron?

 

[Stuart Naylor]

I sort of finally plumbed for this dunno if you guys want to use or grab is so please do.
https://github.com/StuartIanNaylor/log2ram/blob/master/Dev Notes

 

pi@raspberrypi:~ $ stress --vm 6 --vm-bytes 128M --timeout 60s
stress: info: [2693] dispatching hogs: 0 cpu, 0 io, 6 vm, 0 hdd
stress: info: [2693] successful run completed in 61s

On the mighty Pi0

PS just as an opinion I do find how you have locked all your routines into an Armbian only offering sort of crazy and insular.

Crazy as with the wealth of your stuff there should be a plethora of references to installing and using your gear and splashing Armbian strongly across the internet.
zram-config and various over utils would be having Pi owners thinking what is this Armbian thing then... but sadly not.
Dunno who or why but that sort of framework doesn't compute with me, but just opinion and prob to your detriment as much as mine, but hey and cheers for the info...

 

pi@raspberrypi:~ $ zramctl
NAME       ALGORITHM DISKSIZE  DATA  COMPR TOTAL STREAMS MOUNTPOINT
/dev/zram0               160M  2.3M 573.6K 1000K       1 /var/log
/dev/zram1 lz4          54.2M    4K    63B    4K       1 [SWAP]
/dev/zram2 lz4          54.2M    4K    63B    4K       1 [SWAP]
/dev/zram3 lz4          54.2M    4K    63B    4K       1 [SWAP]
/dev/zram4 lz4          54.2M    4K    63B    4K       1 [SWAP]
pi@raspberrypi:~ $ stress --vm 2 --vm-bytes 512M --timeout 60s
stress: info: [906] dispatching hogs: 0 cpu, 0 io, 2 vm, 0 hdd
stress: FAIL: [906] (415) <-- worker 908 got signal 9
stress: WARN: [906] (417) now reaping child worker processes
stress: FAIL: [906] (451) failed run completed in 28s
pi@raspberrypi:~ $

pi@raspberrypi:~ $ zramctl
NAME       ALGORITHM DISKSIZE  DATA  COMPR TOTAL STREAMS MOUNTPOINT
/dev/zram0               160M  2.5M 614.6K    1M       1 /var/log
/dev/zram1 lz4         650.2M    4K    64B    4K       1 [SWAP]
pi@raspberrypi:~ $ stress --vm 2 --vm-bytes 512M --timeout 60s                  stress: info: [837] dispatching hogs: 0 cpu, 0 io, 2 vm, 0 hdd
stress: info: [837] successful run completed in 61s

Same uncompressed mem_limit of 50% of total with a single disk-size of 4 * 54.2M with a disk size expecting 3:1 compression as its a virtual size.  
 

[Stuart Naylor]

What I ended up doing is a grand title of Swapiness load balancer of my usual pretty poor scripting skills but it works
 https://github.com/StuartIanNaylor/zram-swap-config/blob/master/zram-swap-config-slb

If loadavg < 1 then it will creep up to 100 in increments of 5 every 20sec.
if loadavg > 1 it drops back to 80 asap

if loadavg > 2 it lowers to 70 asap

 

I read about the general disinterest in tweaks and tuning as yeah most of the settings are already set to best point for all occasions and manner of operation.
You can tweak some that will suit certain manners of operation.

But you can tweak many to much higher levels if they can react dynamically and back on various forms of process load that they are related to.
Again probably profiles for certain tweaks & tunes will have bias to certain modes of intended operation.

 

Sure you guys could provide far more finesse they I can and release as individual .debs

I stopped flooding azlux/log2ram with zram swap guff and split into 2 so that L2R just does what it says and now has the option for a single zram backing store.

 

 

[GilGonGan]

I just want to thank everyone here for their hard work, while I don't understand most of what's going on, as a mostly Windows dude, I get the impression this is some really cool tuning for some really basic hardware, this whole zram compressed swapfile in ram is a way cool concept.   

 

I have a 512MB Pi Zero and it just works really really fine for basic stuff, thanks to hard work from folks like you all.

(It's a shame I hear newer models run a lot hotter / worse?)  lucky I have an older model.

 

 

[psygnosis]

I'm not expert but I'd like to contribute to test. I'm on Opi1 512Mb RAM and with the last release (Armbian Bionic) there is no swap partition. with 512Mb of DRAM it's unusable (expecially with web browser).

So If I want to contribute and test zram configuration (instead of create a swap partition and automount with fstab), what should I do to start?

thank you in advace

[Igor]

8 hours ago, psygnosis said:

and with the last release (Armbian Bionic) there is no swap partition.

That should not be the case. Please provide logs with:

armbianmonitor -u

 

[BarnA]

Thanks for the really useful discussion here about zram.  I just wanted to add the symptoms I observed with Armbian (Stretch, no GUI) on a 512Mb Orange Pi One which I'm pretty sure (but cannot prove) was due to out of memory conditions, in case this report helps others with the same symptoms.

 

Normal service with Opi1/Armbian for me is to get indefinite uptime (>>6 months).  I was experiencing unexpected crashes after 1-2 weeks and as I run a UPS and am using an M2.SATA SSD to USB adapter its unlikely that powering or sdcard issues are a root cause.

 

When observing the crashes roughly 80% seemed to occur 1-2minutes after the apt.daily timer fired (as reported by 'systemctl list-timers'), the remainder seemed to occur at random.  As I've got an SSD I decided to add 512Mb swap on top of the zram automatically configured.  I set the prioriy of the SSD swap below that of zram and vm.swappiness = 60 (suggestions as to correct setting in these circumstances much appreciated though, I haven't experimented).

 

The system has now been up for approaching 1 month without further outage.  swapon -s shows that just under 20Mb of swap on the SSD is in use.  zram is used heavily.

Filename                                Type               Size       Used       Priority
/swapfile1                              file                 524284  18308    2
/dev/zram1                             partition        252000  198692  5

 

I think that for users with SSD like me it may be useful to supplement zram with normal swap in the case where they have a device with a small amount of memory such as an Opi1.  Comparing Armbian Buster to Stretch memory footprint for my use case, I wonder if this may become more important going forward?