Espressobin support development efforts

[tkaiser]

On 8/23/2018 at 7:05 PM, danglin said:

Here are results for 800 and 1000:

http://ix.io/1l2a

http://ix.io/1l2n

 

And here's for the flash-image-1g-2cs-1200_750_boot_sd_and_usb.bin BLOB: http://ix.io/1lCe (if you look closely you see that between 23:01:08 and 23:06:24 some background activity happened -- one of my Macs backing up to the EspressoBin -- so that I had to repeat the OpenSSL test on an idle system later)

 

With 'working' cpufreq my EspressoBin idles with 200 MHz at 5.8W (measured at the wall) and the SoC is hot like hell. Now without CONFIG_ARM_ARMADA_37XX_CPUFREQ and the 1200 MHz settings the board idles at 6.5W while running all the time at 1190 MHz, still being hot like hell.

 

A difference of 0.7W is a joke given that these idle numbers are way too high anyway. There's only one SATA HDD connected (standby) and one LAN connection. RockPro64 with same 12V PSU measures below 3.7W.

 

Update: with flash-image-1g-2cs-800_800_boot_sd_and_usb.bin the board idles at 5.9W.

[sunxishan]

I am curious why not the cifs support built in release.

 

linux-mvebu64-default.config: CONFIG_CIFS=m

linux-mvebu64-dev.config:# CONFIG_CIFS is not set

linux-mvebu64-next.config:# CONFIG_CIFS is not set

 

should we have this basic module at lease CONFIG_CIFS=m as in default config? 

 

[lanefu]

I am curious why not the cifs support built in release.
 
linux-mvebu64-default.config: CONFIG_CIFS=m
linux-mvebu64-dev.config:# CONFIG_CIFS is not set
linux-mvebu64-next.config:# CONFIG_CIFS is not set
 
should we have this basic module at lease CONFIG_CIFS=m as in default config? 
 

Effecticely cifs is deprecated. Using SMB over cifs is standard practice these days.

https://blog.varonis.com/cifs-vs-smb/

Sent from my SM-G950U using Tapatalk

[tkaiser]

46 minutes ago, lanefu said:

Effecticely cifs is deprecated

 

Which doesn't change much wrt the name of Linux kernel modules that provide SMB3 client functionality  

 

Speaking about Linux naming conventions: 'mount_smb' is the deprecated variant of 'mount_cifs' that should be used today. And this most probably originated from the history of SMB/CIFS support in Linux. Two decades ago SMB could've been best described as a pile of rotten protocols that are pretty much useless since Microsoft's implementations differed in almost any detail. Compared to that CIFS was an advancement (read through Linux manual pages -- many of this historical stuff still there). SMB2 and SMB3 have nothing in common with the SMB we know from 2 decades ago. Robust protocols with lots of cool features and specifications worth the name.

 

Anyway: CONFIG_CIFS is not set on just 5 kernel variants (by accident) so @sunxishan please send a PR with them enabled as module.

[sunxishan]

On 8/31/2018 at 7:32 AM, tkaiser said:

 

Which doesn't change much wrt the name of Linux kernel modules that provide SMB3 client functionality  

 

Speaking about Linux naming conventions: 'mount_smb' is the deprecated variant of 'mount_cifs' that should be used today. And this most probably originated from the history of SMB/CIFS support in Linux. Two decades ago SMB could've been best described as a pile of rotten protocols that are pretty much useless since Microsoft's implementations differed in almost any detail. Compared to that CIFS was an advancement (read through Linux manual pages -- many of this historical stuff still there). SMB2 and SMB3 have nothing in common with the SMB we know from 2 decades ago. Robust protocols with lots of cool features and specifications worth the name.

 

Anyway: CONFIG_CIFS is not set on just 5 kernel variants (by accident) so @sunxishan please send a PR with them enabled as module.

Hi tkaiser thanks for your reply. I would love to submit the PR however when I checked the config file it labelled as auto-generated file. I am not familiar with this building procedure how I generate this config file. 

 

[tkaiser]

2 hours ago, sunxishan said:

I am not familiar with this building procedure how I generate this config file

 

Hopefully fixed with https://github.com/armbian/build/commit/dc9ad0e1e5993cbe92bbbe25417a419fa1fc1123

[sunxishan]

19 hours ago, tkaiser said:

 

Hopefully fixed with https://github.com/armbian/build/commit/dc9ad0e1e5993cbe92bbbe25417a419fa1fc1123

Thanks a lot! 

[ebin-dev]

There are some new bootloader flash images  for the EspressoBin (see the parallel thread).

 

U-Boot 2017.03-armada-18.09.1-ga92bd86-armbian

WTMI-devel-18.07.0-6050fd5

atfv1.5(release):711ecd3 (Marvell-armada-18.09.4)

 

It is mandatory to reset the environment settings and to use the new ones (load addresses have changed). 

 

Edit: 18.09 images with slightly improved performance: sbc-bench/17.10 vs. sbc-bench/18.09

[y52]

On 9/6/2018 at 11:39 AM, ebin-dev said:

It is mandatory to reset the environment settings and to use the new ones (load addresses have changed). 

I've upgraded the u-boot loader to the last one :

U-Boot 2017.03-armada-18.09.1-ga92bd86-armbian (Sep 05 2018 - 21:49:34 +0200)

Model: Marvell Armada 3720 Community Board ESPRESSOBin
       CPU     1000 [MHz]
       L2      1000 [MHz]
       NB AXI  250 [MHz]
       SB AXI  250 [MHz]
       DDR     800 [MHz]

 

Then I have changed the u-boot environments as it was required.  Much to my surprise, I was unable to boot initially.

I boot from the the 'mmc' device and the logs showed the errors :

Marvell>> 
1176 bytes read in 13 ms (87.9 KiB/s)
## Executing script at 06d00000
221 bytes read in 3 ms (71.3 KiB/s)
** File not found Image **       
5837603 bytes read in 256 ms (21.7 MiB/s)
** File not found boot/dtb/marvell/armada-3720-community.dtb **
8330 bytes read in 10 ms (813.5 KiB/s)
Bad Linux ARM64 Image magic!

 

After some debugging, I found, that the 'image_name' environment is badly formed. Indeed : 

Marvell>> printenv image_name
image_name=Image

while the image file is located in 'boot/Image'

 

I tried changing the 'boot_prefixes' environment as follows:

Marvell>> setenv boot_prefixes '/boot/'
Marvell>> printenv boot_prefixes       
boot_prefixes=/boot/

But it didn't help.

 

So I set :
setenv  image_name boot/Image

saveenv

 

The boot went fine until the next 'shutdown -r now'. When hot rebooting again, the boot process seems to process the sequence for a different boot target :  

 

Starting kernel ...

[    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd034]
[    0.000000] Linux version 4.17.9-mvebu64 (root@nightly) (gcc version 7.2.1 20171011 (Linaro GCC 7.2-2017.11)) #164 SMP PREEMPT Tue Jul 24 18:15:33 UTC 2018
[    0.000000] Machine model: Globalscale Marvell ESPRESSOBin Board
[    0.000000] earlycon: ar3700_uart0 at MMIO 0x00000000d0012000 (options '')
[    0.000000] bootconsole [ar3700_uart0] enabled
Loading, please wait...
starting version 237
Begin: Loading essential drivers ... done.
Begin: Running /scripts/init-premount ... done.
Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done.
Begin: Running /scripts/local-premount ... Scanning for Btrfs filesystems
done.
Begin: Waiting for root file system ... Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... done.

....many times....

Begin: Running /scripts/local-block ... done.
Begin: Running /scripts/local-block ... mdadm: error opening /dev/md?*: No such file or directory
/scripts/local-block/mdadm: 58: /scripts/local-block/mdadm: rm: not found
done.
....many times....
Begin: Running /scripts/local-block ... done.
done.
Gave up waiting for root file system device.  Common problems:
 - Boot args (cat /proc/cmdline)
   - Check rootdelay= (did the system wait long enough?)
 - Missing modules (cat /proc/modules; ls /dev)
ALERT!  /dev/mmcblk0p1 does not exist.  Dropping to a shell!
(initramfs) 

 

 

 

If I unplug the board and power it back again, the board boots normally.

What's wrong with the u-boot scripts and environments?

 

Besides, is the new kernel build 4.18.xx available for upgrade?

 

[ebin-dev]

"Updated u-boot needs new default boot environment and new boot script (overwrite the one on your /boot media – needed only if you upgrade from < v5.59)". (see the EspressoBin download page)

 

[y52]

1 hour ago, ebin-dev said:

new boot script (overwrite the one on your /boot media – needed only if you upgrade from < v5.59)". (see the EspressoBin download page)

I am still on ARMBIAN 5.44 user-built Debian GNU/Linux 9 (stretch) 4.17.9-mvebu64.

I've used the new u-boot environments provided in the the EspressoBin download page), but there are no instructions for the changes in the /boot media.

Could you be more specific about the new /boot scripts?

[umiddelb]

The new firmware prohibits the address usage between 0x4000000 and 0x54f0000, therefore you need to adjust the load addresses for initrd, kernel and dtb.   

[y52]

3 hours ago, umiddelb said:

you need to adjust the load addresses for initrd, kernel and dtb.   

The load address is not an issue. It has been set correctly :

Marvell>> setenv fdt_addr 0x6000000
Marvell>> setenv kernel_addr 0x7000000
Marvell>> setenv loadaddr 0x8000000
Marvell>> setenv initrd_size 0x2000000
Marvell>> setenv initrd_addr 0x1100000
Marvell>> setenv scriptaddr 0x6d00000

 

What is broken in the new u-boot environment is the path to the 'image_name'

image_name=Image

while the image file is located in 'boot/Image'

and the u-boot command

"scan_dev_for_boot 'for prefix in ${boot_prefixes}; do echo ${prefix};run boot_a_script; done'" doesn't return the correct path to the Image file.

There are probably other issues, as the board doesn't boot consistently between cold boot and hot reset.

[y52]

6 hours ago, ebin-dev said:

new boot script (overwrite the one on your /boot media – needed only if you upgrade from < v5.59)". (see the EspressoBin download page)

I've found the new boot script in

https://dl.armbian.com/espressobin/u-boot/bootscript/

 

However, looking at their content, it seems, that boot.cmd and boot.scr are unrelated to each other and different.

As I understand it, boot.scr should be compiled from boot.cmd : "mkimage -C none -A arm -T script -d /boot/boot.cmd /boot/boot.scr".

Why are they completely different? Is there any confusion?

boot.scr refers to display environments like setenv disp_mode "1920x1080m60" etc.

It should be coming from another build and another board.

Which one is correct and should be used for expressobin? boot.cmd looks to be more appropriate, but should be recompiled before used with the boot.

[y52]

Interestingly enough, that CPU scaling seems not being reported by armbianmonitor :

14:54:44: 1000MHz  0.00   0%   0%   0%   0%   0%   0%
14:54:49: 1000MHz  0.00   3%   0%   2%   0%   0%   0%
14:54:54: 1000MHz  0.00   1%   0%   0%   0%   0%   0%
14:54:59: 1000MHz  0.00   0%   0%   0%   0%   0%   0%
14:55:05: 1000MHz  0.00   1%   0%   0%   0%   0%   0%
14:55:10: 1000MHz  0.00   0%   0%   0%   0%   0%   0%
14:55:15: 1000MHz  0.00   1%   0%   0%   0%   0%   0%

or is broken with this U-Boot 2017.03-armada-18.09.1-ga92bd86-armbian (Sep 05 2018 - 21:49:34 +0200).

The previous u-boot versions gave the impression of a functional scaling :

12:32:23: 1000MHz  1.01  21%   7%  10%   0%   4%   0%
12:32:28: 1000MHz  0.93   2%   1%   0%   0%   0%   0%
12:32:33:    0MHz  0.86   1%   0%   0%   0%   0%   0%
12:32:38:    0MHz  0.79   2%   1%   0%   0%   0%   0%^C

 

[y52]

3 hours ago, y52 said:

new boot script in

https://dl.armbian.com/espressobin/u-boot/bootscript/

 

However, looking at their content, it seems, that boot.cmd and boot.scr are unrelated to each other and different.

As I understand it, boot.scr should be compiled from boot.cmd : "mkimage -C none -A arm -T script -d /boot/boot.cmd /boot/boot.scr".

I've recompiled  boot.scr from boot.cmd and, this time,  the boot with the U-Boot 2017.03-armada-18.09.1-ga92bd86-armbian (Sep 05 2018 - 21:49:34 +0200) went without a hitch.

Still the absence of CPU scaling persists.

I shall observe how stable the board is with this U-Boot.  The previous configuration resulted in hazardous reboots each couple of days.

[ebin-dev]

6 hours ago, y52 said:

What is broken in the new u-boot environment is the path to the 'image_name'

image_name=Image

while the image file is located in 'boot/Image'

and the u-boot command

Have you seen this https://github.com/armbian/build/commit/9c7ce48f2b6cef71080ee6f04cca6f521f98df96 ?

53 minutes ago, y52 said:

Still the absence of CPU scaling persists.

Have you tried # cat /sys/devices/system/cpu/cpu1/cpufreq/stats/trans_table ?

53 minutes ago, y52 said:

The previous configuration resulted in hazardous reboots each couple of days.

I do remember that you should have returned your broken board.

[y52]

48 minutes ago, ebin-dev said:

7 hours ago, y52 said:

and the u-boot command

Have you seen this https://github.com/armbian/build/commit/9c7ce48f2b6cef71080ee6f04cca6f521f98df96 ?

Yes. I understood it. It goes in line with the changes made to boot.cmd

You should still replace the wrong boot.scr on your download page.

48 minutes ago, ebin-dev said:

1 hour ago, y52 said:

Still the absence of CPU scaling persists.

Have you tried # cat /sys/devices/system/cpu/cpu1/cpufreq/stats/trans_table ?

I have this table in place :

root@karmae:~# cat /sys/devices/system/cpu/cpu1/cpufreq/stats/trans_table 
   From  :    To
         :    200000    250000    500000   1000000 
   200000:         0         1         2         0 
   250000:         1         0      1846      3910 
   500000:         2      1822         0        85 
  1000000:         0      3934        61         0 

What should be necessary to check more ?

 

48 minutes ago, ebin-dev said:

I do remember that you should have returned your broken board.

globalscale technologies turns a deaf year to such claims. They consider their boards to fully meet the specifications.  I could have sent the board at my own risk for the cost of 50 USD, which is not worth it.   globalscale technologies doesn't even guaranty the replacement. Once, the RMA guy told, that they may check it, but since then he doesn't even dare responding. It is really not up to the mark handling so badly technical claims for their boards.

[Igor]

22 hours ago, y52 said:

You should still replace the wrong boot.scr on your download page.

Fixed.

[sunxishan]

Another quick question: 

Is there anyway that we can use the second UART port in P9 connector? I know something need to change to generate new dtb file to enable that. Is there any easy way to download or generate this file? 

 

Thanks a lot. 

 

p.s. digging on the internet and found this: 

https://patchwork.kernel.org/patch/9988925/

 

The reason why UART is not enabled: "... because both headers are dedicated to expose general purpose pins and remapping some of them to use the second UART would break existing users."

 

However, after checking the GPIO mapping, I found PIN 24&26 on P9 header are used for UART only, not possibly messing around with other GPIO pins. It should be opened at the beginning.

 

Can we have a dts file with this option on?  Maybe I have to learn how to compile the dts file myself. 

 

 

[Pavel Odintsov]

Hello!

 

Thank you for porting Ubuntu 18.04 to EspressoBin! It works great  

 

I noticed only small issues with systremd:

[FAILED] Failed to start Wait for Network to be Configured.
See 'systemctl status systemd-networkd-wait-online.service' for details.

 

[Pavel Odintsov]

Well, I noticed one issues with this kernel.  Any chance to enable kernel configuration option CONFIG_XDP_SOCKETS in kernel for EspressoBin? We participated in testing this feature when it was in bpf-next and it worked fine on our ARM64 machines (Thunder X machines from Cavium). I expect that it should work on EspressoBin fine too.

 

I tried test app for AF_XDP and it does not work:

libbpf: failed to create map (name: 'xsks_map'): Invalid argument
libbpf: failed to load object './xdpsock_user_kern.o'

And dump from strace:

 

2909  bpf(BPF_MAP_CREATE, {map_type=BPF_MAP_TYPE_ARRAY, key_size=4, value_size=4
, max_entries=1, map_flags=0, inner_map_fd=0, ...}, 72) = 3
2909  bpf(BPF_MAP_CREATE, {map_type=0x11 /* BPF_MAP_TYPE_??? */, key_size=4, val
ue_size=4, max_entries=4, map_flags=0, inner_map_fd=0, ...}, 72) = -1 EINVAL (In
valid argument)
2909  write(2, "libbpf: failed to create map (name: 'xsks_map'): Invalid argumen
t\n", 66) = 66
2909  close(3)                          = 0
2909  write(2, "libbpf: failed to load object './xdpsock_user_kern.o'\n", 54) =
54

It offers significantly improved logic for network processing and will be useful for network targeted boards like this. 

 

Thank you!

[Pavel Odintsov]

I created PR to enable this feature: https://github.com/armbian/build/pull/1146 

[Pavel Odintsov]

Thank you for merging! 

 

I prepared kernel with AF_XDP enabled and my test application works fine:

 sock0@wan:0 rxdrop 	
                pps         pkts        1.00       
rx              9,310       9,323      
tx              0           0          

Packet rate aren't impressive but my packet generator works over WIFI and Powerline. I do not expect performance here 

 

That's great place for future improvements! 

[danglin]

On 9/9/2018 at 6:54 AM, y52 said:

There are probably other issues, as the board doesn't boot consistently between cold boot and hot reset. 

Adding "rootdelay=3" to bootargs helped reboot with SanDisk Extreme SD card.  With just "rootwait", the kernel sometimes tries

to mount the root file system before the SD card is ready.

[y52]

9 hours ago, danglin said:

Adding "rootdelay=3" to bootargs

Good suggestion. Which file the boot arguments are set? I looked into armbianEnv.txt, but "rootwait" is not there.  

Otherwise, are you suggesting changing the variable in the bootloader ?

I looked into the current U-Boot 2017.03-armada-18.09.1-ga92bd86-armbian

last Marvell>> printenv

the rootarg is not set either.

Could you be more specific?

[danglin]

6 hours ago, y52 said:

Good suggestion. Which file the boot arguments are set? I looked into armbianEnv.txt, but "rootwait" is not there.  

Otherwise, are you suggesting changing the variable in the bootloader ?

I looked into the current U-Boot 2017.03-armada-18.09.1-ga92bd86-armbian

last Marvell>> printenv

the rootarg is not set either.

Could you be more specific?

I changed it in the /boot/boot.cmd file:

setenv bootargs "$console root=${rootdev} rootfstype=${rootfstype} rootdelay=3 rootwait resume=none loglevel=${verbosity} usb-storage.quirks=${usbstoragequirks} mtdparts=spi0.0:1536k(uboot),64k(uboot-environment),-(reserved) ${extraargs}"
echo bootargs=${bootargs}

 

Then you need to run "mkimage -C none -A arm -T script -d /boot/boot.cmd /boot/boot.scr" to regenerate /boot/boot.scr.

 

You might want to make a backup copy of /boot/boot.scr first so that it's possible to recover easily.  Since this is an armbian file, it might

get updated by apt.

 

It looks like you could add it to "extraargs" in armbianEnv.txt, or in the u-boot environment.

 

 

[danglin]

16 minutes ago, danglin said:

It looks like you could add it to "extraargs" in armbianEnv.txt, or in the u-boot environment.

The former works.  Add "extraargs=rootdelay=3" to armbianEnv.txt for a delay of 3 seconds

[y52]

On 11/23/2018 at 4:29 PM, danglin said:

Add "extraargs=rootdelay=3" to armbianEnv.txt

I added this setting to give (boot log):

[    0.000000] Kernel command line: console=ttyMV0,115200 earlycon=ar3700_uart,0xd0012000 root=UUID=cdf86afe-e235-4b45-a02b-7954f2bbdaaa rootfstype=ext4 rootwait loglevel=1 usb-storage.quirks=0x2537:0x1066:u,0x2537:0x1068:u mtdparts=spi0.0:1536k(uboot),64k(uboot-environment),-(reserved) rootdelay=3
[    0.000000] Memory: 2026008K/2097152K available (10684K kernel code, 730K rwdata, 3244K rodata, 640K init, 502K bss, 54760K reserved, 16384K cma-reserved)
 

 

However, the board didn't show up after "shutdown -r now". It was still necessary unplugging it and power up back for it to reboot. "rootwait" and "rootdelay=3", are those rootargs not conflicting and self excluding?

[danglin]

6 hours ago, y52 said:

However, the board didn't show up after "shutdown -r now". It was still necessary unplugging it and power up back for it to reboot. "rootwait" and "rootdelay=3", are those rootargs not conflicting and self excluding?

I mentioned specifically that this helped with a SanDisk Extreme microSD card.  I believe there are multiple problems and

some SD cards don't work well.  With a Team TUSDH32GUHS03, I consistently get a mmc error on hot reboots (0x208000).

According to the Marvell 88F3700 Functional Specs, this is a read data CRC error.  I've also seen u-boot fail in memory

training but this doesn't happen very often.  Strangely, the Team card works fine after the board boots.
 

With the Team card, read errors occur first reading the Armbian boot.scr file, and later reading the Image, uInitrd and dtb files.

 

With the SanDisk card, Linux would start but sometimes the root file system would fail to mount.  I don't see CRC errors with it.

Since I added rootdelay, the board has booted successfully every time.  Three seconds is the minimum for my card.

 

Armbian recommends Samsung and SanDisk brands.  If you search Internet, you will find this is a common problem.  They

weren't designed for random access applications, and I believe they lack TRIMM support.

 

I don't believe the options are conflicting and self excluding.  It doesn't say that in the kernel documentation.  Nominally,

rootwait should cause the kernel to wait until the root file system is mounted.  But, it seems trying to mount it before the

SD card is ready causes problems.  The rootdelay option simply delays mounting the root file system for 3 seconds.  You

can see delay in mounting root after Linux starts.  The delay won't fix the mmc read error problem.