Matthias

Hallo, I've been using Armbian on the NanoPi M4 V2 for a while and now I'm thinking about switchen from the legacy kernel to the current kernel. Unfortunately that's not so easy to do when using armbian-config. You could think you can just select the current-kernel in the list but you won't find it there. The reason is simple: the legacy kernel belongs to the rk3399-family and the current kernel is rockchip64. Which is basically the same, but from an organisational point of view it is not (please forgive me if I do not show an adequate technical precision here, the details and the plan to clean that up are explained here). What I would be interested in is now: Is there a proper way to switch the kernel manually? As I said, I have been using the system for a while now, so setting it up from scratch with the right kernel is something I would like to avoid. I had a look into the source of armbian-config so I know that manipulating the LINUXFAMILY in /etc/armbian-release will display the current-kernel in the list. But I don't want to imagine what kind of side-effects will hit me when starting the installation process that way. Cheers, Matthias

I played around with setting the governor to "conservative" or "ondemand". Result: "conservative" results in a stable system that allows copying of several GBs of data, "ondemand" results in an unstable system that fails after copying some GBs of data and corrupts its filesystem I also did a scripted load test that copied 100GB (fast: data source was tmpfs) of data to the hard drives with governor set to "conservative". Result: No reported file system errors and a MD5-sum-check of every file copied to the disks passed successfully. If anybody is interested and likes to repeat that test on his configuration I attached the script. It's currently made for BTRFS. If you are using a different file system, just remove the part with the scrubbing. load_test.sh

And repeated the copying with kernel 5.4.11. The CPU frequencies were limited in the sysfs (govenor "conservative", scaling_max_freq) to 1.42/1.8GHz. I copied 30GB of data to the hard drive (this time BTRFS) and got the first errors (file system repors CRC-errors) while calculating the MD5-sums of the copied files. My conclusion: No indicators that the CPU-frequency spoils the data, but something seems to be odd with kernel 5.4.x and NanoPi M4V2. Or do you think the frequency needs to be reduced the hard way in the kernel? @piter75 Just noticed I only used the governor "conservative" for the A72, the A53 was set to "on demand". Frequencies were correct. So I need to recall my conclusion. I repeated the test, this time all CPUs were set to "on demand". And voilà: All data was written correctly. Strange. I don't have the time right now to go back and repeat the test again with the "on demand" governor to see that the problems appear again, but so far my conclusion is: The NanoPi M4V2 only runs reliable if the CPU frequencies are limited to 1.42/1.8GHz AND the governor is NOT set to "on demand" ("conservate" seems to work).

Just did some copying of files using kernel 4.4.210: I copied around about 15GB of random data and about the same amount of other data (videos) on a ZFS-volume. Also copied about 5GB of data from SD to the ZFS-volume. MD5-checks indicate that there has not been any data corrupted. Cpufreq shows the maximum allowed frequency of the CPUs is 1.42/1.8GHz. So the combination of kernel 4.4.x and "low" CPU frequency seems to do the trick. I'll try again with a 5.4.x-kernel and throttled CPUs.

Don't know exactly, I think it was 4.4.192.

I didn't observe any crashes because of high load but what I encountered when compiling code were messages on the console indicating (indirectly) that there is something odd with the CPU (sorry, I don't remember the text). I tried to fix that using more powerful power supply, but that did not help. Not even the 12V supply via the SATA-hat. What did help was setting the governor for the CPU frequency to "conservative". Since I did that, I never got any strange messages anymore. So my conclusion was, that the board might have problems handling spikes or large changes in power consumption. But of course, that's not a prove against crashes...

Don't worry. I remember darkly that when I tested it one month ago the legacy kernel was good at receiving and bad at sending while the current kernel was cood at sending and bad at receiving. See here:

Interesting, I also had problems copying files using the 5.4 kernel on my NanoPi M4V2. I compiled it myself and copies files from the sd-card to and external hard drive connected via the SATA-hat. I used file systems on the hard drive that were able to detect corrupted data on the drive (ZFS and BTRFS) and every time I did a scrub after copying the files, failed CRC-checks within the data were reported, some of them even at the same position on two redundant drives (mirror). I'll try again with the legacy kernel (4.x). Would be greate if it works in this configuration. @piter75: Did you apply your fixes for ethernet also to the legacy kernel? Then it would be much easier to get the data onto the device in the first place.

That would explain some things. For example: During scrubbing a mirrored BTRFS reported broken CRCs of the same block of data on both disks. I observed that more than once within 40gb of data. Chances of this happening by accident are of course very low. But if the data gets corrupted at the source (RAM) this could easily happen. Configuring the timing of the RAM is way beyond my knowledge, so I'll stop here and wait for any improvements. As always: If there is something you would like to get tested: Don't hesitate to contact me.

I need to recall my thesis about ZFS interfering with Samba and causing trouble. I cannot reproduce it anymore. For testing with NFS I installed the NFS client for Windows 10 wich seemed to interfere a lot with SMB. I don't know why, but after uninstalling it, everything was fine and I was able to download and upload files in large amounts. Of course having the offloading disabled. This is true for transferring data from and to the SD card. When using a hard drive connected to the SATA hat I get stopping network connections, broken file systems (ZFS and BTRFS two-disk-mirrors that show uncorrectable errors without any forced reboot) and kernel panics. But that seems to be a different story (broken hardware? temperature?) and does not have a direct connection to Samba or the network connection. Long story short: I consider the Samba problem to be solved and need to continue examining my hardware setup.

Ok @piter75, looks like things are a little more complicated: First of all I must apologise: My "fresh install" had at least one customisation I did not mention: The kernel had support for ZFS (via zfs-dkms). When using this installation Piter's use of ethtool does not bring any improvement. Then I remembered something: ZFS brings has support for SMB and NFS. Don't know why they mixed that but seems to be a relict from Solaris. So just to make sure I reinstalled the system without ZFS-support. The following observations were made using the "ZFS-less" system connected to a Windows 10 PC via gigabit ethernet: Copying a lot of data via SMB to and from the share hosted on the NanoPi without "sudo ethtool -K eth0 tx off rx off": After some gigabytes the transfer speed drops to zero and the NanoPi gets unresponsible. It looked like the green LED changed it's flashing-pattern so it was flashing faster, but I might be wrong. Doing the same with "sudo ethtool -K eth0 tx off rx off": I successfully copied about 20GB to and from the NanoPi. No problems detected. So I think this change brings improvement. And combining Samba and ZFS does not seem to be a good idea, at least not for the versions I used (kernel 5.4.10, Samba 4.9.5, zfs-dkms 8.2.3). For whatever reason. Shall we add the call of ethtool to the build process of Armbian images for the NanoPi M4V2?

That's good news, thank you for that hint, Piter. And good to see that my issue is reproducible on more than one board. I also read about that a couple of weeks ago. But as that was the time where we all struggled with the stability of the networking, it didn't bring any effect and I dropped that idea. I can give it another try this evening on the current state of the networking. Hopefully it has the effect I'm looking for.

Checked NFS: Works fine, I was not able to observe any drops of transfer speed like mentioned above.

I can check NFS, that's a good idea. Then I get a better impressing how "strongly" related the issue is to Samba. If Samba works on one board family and not on another, the only difference relevant to Samba I can think of is the kernel. Or did I miss something?

Edit: Issue is solved, see here. It's an application problem but according to an extensive Google research this does not seem to be a common problem of Samba, so it might be an integration problem with the environment it is running in. The setup: Armbian 19.11.6 (Debian Buster) running the current kernel on a NanoPi M4V2, offering network shares via Samba. Clients are Windows 10 and a BananaPi running Armbian (Debian Buster 4.19.y kernel) as well. The observation: When downloading (large) files from the Samba share to Windows 10, the download speed very often suddenly drops down to zero and stays on that rate (forever). This can be best observed on files of several GB size as it usually takes some seconds until the problem appears. There seems to be a correlation with the download speed: Using ethernet it happens more often than when using slower WiFi. I also increased the log level of Samba but I could not detect anything special the moment the rate drops down (In the Samba logs. There are no significant lines in /var/log/messages or /var/log/syslog). The transfer just stops. The same behavior can be observed when mounting the share on the BananaPi and then copy large files. My first guess was a generally unstable network connection as there have been committed some fixes in this area to Armbian recently. But when stressing the network via iperf3 in both directions I could not detect any variations in the transfer speed. In both directions. When writing data to the Samba shares I do not observe any problems. I was able to upload about 30GB without any significant drop in speed. I also took the working smb.conf from my BananaPi, which serves files flawlessly via Samba, adapted it and ran tests. But I observed the same behavior. So I assume there is a problem the way Samba accesses the files or uses the network stack in the Armbian environment. But as this is only speculation and I did not find any further indicators for the source of the problem I now hope that somebody else stumbled over the same thing and maybe already has a solution for it. If somebody likes to reproduce it: Take a fresh install, install Samba add a share to a directly on the SD card in smb.conf (no further changes in the config file). Then copy a file of about 3GB to the share. Then try to download the file again. You will be impressed by the transfer speed (as long as the file is cached in RAM on the NanoPi M4V2) of more than 100MB/s but then the transfer suddenly stops.