Espressobin support development efforts

[barish]

On 10/3/2019 at 8:55 PM, mu-b said:

I'm asking GlobalScape if they know the root cause, that is the component that is not functioning correctly, wonder if I will receive of a response, Netgate don't seem to want to say.

I hope you will - looking forward to their response!

[mu-b]

Well I've fixed my instability issue. The root cause was the same as https://forum.netgate.com/topic/144636/sg-1100-intermittent-reboots, and it was power related. The fix was to replace the 470uf/16V capacitor (component EC1 on the schematic) sat immediately behind the 12V DC jack which under testing leaked under certain temperature conditions (variations). As such the board is now stable and has been running OpenWRT 18.06.2 for 24 hours with memtester and stress running continuous during that entire time.

 

I'll leave it a few more days, after that I'm going to assume that its fixed and that is what NetGate were referring too when they mention 'a power related component'.

 

And GlobalScape will not tell you, they know of the problem, in a reply from Kevin Liu he says 'Yes, I do have knowledge regards the power related component issue.' when I asked which component it was so i could attempt to replace myself.

[FlashBurn]

I found some time and tried the current version on my board:

 

http://ix.io/26iy

 

Still the same problem that the frequency is the wrong one

 

Am I the only one who is bothered by this? Does it work right for others?

[Anders]

@FlashBurn I'm having the same problem - only getting 800MHz. It's frustrating, since I bought this board because it claimed to deliver 1.2GHz.

 

I think this is a problem in Linux, but I'm not sure where to report it, so I made a post in the espressobin.net forums.

Now I just discovered that they deleted my thread, but here's a repost:

 

Edited February 23, 2020 by Anders
Update link.

[Smurfix]

Yet another reason not to buy one of those things.

[lanefu]

19 minutes ago, Smurfix said:

Yet another reason not to buy one of those things.

Yeah you guys are inspiring me to just throw my 2 boards in the trash.  I wanted to build an OMV box for my brother-in-law but I'm afraid to do so.

[Smurfix]

To be fair, I have three, from the first series. With identical settings, one is definitely unstable (dunno how they got it to pass what Globalscale considers testing, let alone what I would do), one is 100% stable and working as my main router, and one is in my "TODO try with current mainline" pile. We'll see whether I can get the thing to work without much hassle; if not, it'll be replaced with a Raspberry Pi 4 and two USB3 Ethernet dongles.

 

NB: Recycling, not trash, please. (Yes I know that electronics recycling is far from optimal.)

[barish]

Despite of all the seemingly bad news, I boldly ordered a batch of 10 EspressoBin V7 recently. Globalscale promised to test them for me for stability (I am not interested in squeezing out the last bit of performance), and I tested them again after reception: They all seem to run stable with Ubuntu Xenial (the "newest" image provided by Globalscale) and with Armbian kernel 4.19.59-mvebu64.

Since there is still a few weeks till they go on duty, if anyone is interested in a particular test on them, I am willing to do so. I am not into kernel development, so I would need some instruction. ;-)

[lanefu]

14 hours ago, barish said:

Despite of all the seemingly bad news, I boldly ordered a batch of 10 EspressoBin V7 recently. Globalscale promised to test them for me for stability (I am not interested in squeezing out the last bit of performance), and I tested them again after reception: They all seem to run stable with Ubuntu Xenial (the "newest" image provided by Globalscale) and with Armbian kernel 4.19.59-mvebu64.

Since there is still a few weeks till they go on duty, if anyone is interested in a particular test on them, I am willing to do so. I am not into kernel development, so I would need some instruction. ;-)

 

that is bold! I hope it goes smoothly..

You could try the Armbian testing tools.. its just a few config files to adjust for your use..

https://github.com/armbian/autotests

[Andrius]

After a power outage my espressobin does not boot from SD card. In my kermit console I see:

## Executing script at 06d00000
Wrong image format for "source" command
/boot/
1119 bytes read in 16 ms (67.4 KiB/s)
## Executing script at 06d00000
221 bytes read in 5 ms (43 KiB/s)
16488960 bytes read in 708 ms (22.2 MiB/s)
8029703 bytes read in 353 ms (21.7 MiB/s)
** File not found /boot/dtb/marvell/armada-3720-community.dtb **
8942 bytes read in 9 ms (969.7 KiB/s)
## Loading init Ramdisk from Legacy Image at 01100000 ...
   Image Name:   uInitrd
   Image Type:   AArch64 Linux RAMDisk Image (gzip compressed)
   Data Size:    8029639 Bytes = 7.7 MiB
   Load Address: 00000000
   Entry Point:  00000000
   Verifying Checksum ... OK
## Flattened Device Tree blob at 06000000
   Booting using the fdt blob at 0x6000000
   Loading Ramdisk to 3ee82000, end 3f62a5c7 ... OK
   Using Device Tree in place at 0000000006000000, end 00000000060052ed

Starting kernel ...

[    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd034]
[    0.000000] Linux version 4.19.56-mvebu64 (root@armbian.com) (gcc version 7.4.1 20181213 [linaro-7.4-2019.02 revision 56ec6f6b99cc167ff0c2f8e1a2eed33b1edc85d4] (Linaro GCC 7.4-2019.02)) #5.89 SMP PREEMPT Tue Jun 25 22:27:47 CEST 2019
[    0.000000] Machine model: Globalscale Marvell ESPRESSOBin Board
[    0.000000] earlycon: ar3700_uart0 at MMIO 0x00000000d0012000 (options '')
[    0.000000] bootconsole [ar3700_uart0] enabled
Loading, please wait...
starting version 237
Begin: Loading essential drivers ... done.
Begin: Running /scripts/init-premount ... done.
Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done.
Begin: Running /scripts/local-premount ... Scanning for Btrfs filesystems
done.
Begin: Waiting for root file system ... Begin: Running /scripts/local-block ... done.
done.
Gave up waiting for root file system device.  Common problems:
 - Boot args (cat /proc/cmdline)
   - Check rootdelay= (did the system wait long enough?)
 - Missing modules (cat /proc/modules; ls /dev)
ALERT!  /dev/mmcblk0p1 does not exist.  Dropping to a shell!


BusyBox v1.27.2 (Ubuntu 1:1.27.2-2ubuntu3.2) built-in shell (ash)
Enter 'help' for a list of built-in commands.

(initramfs) 

Indeed, there is no device /dev/mmcblk0p1. But I can see the bootable sd card at /dev/mmcblk1p1.

Any ideas what is wrong here?

[Werner]

36 minutes ago, Andrius said:

After a power outage my espressobin does not boot from SD card. In my kermit console I see:

Indeed, there is no device /dev/mmcblk0p1. But I can see the bootable sd card at /dev/mmcblk1p1.

Any ideas what is wrong here?

Maybe the filesystem got corrupted. Can you mount it the sd card on another computer?

[Andrius]

I can mount it on other computer and even on the espressobin initramfs shell. Espressobin sees it as mmcblk1p1.

Edited June 13, 2020 by Andrius

[martinayotte]

2 hours ago, Andrius said:

Any ideas what is wrong here?

Since you are able to mount it elsewhere, check its /etc/fstab, if rootfs is defined as /dev/mmcblk0p1, change it to /dev/mmcblk1p1 and try to boot it again ...

[barish]

As a workaround, try putting the following line into /boot/armbianEnv.txt:

rootdev="/dev/mmcblk1p1"

Or modify and compile boot.cmd accordingly. Why the SD card's device has changed though, I cannot tell. I know that Linux changes device names for HDDs, which might be called /dev/sda or /dev/sdb with no apparent hardware or software change.

[barish]

Can anyone tell me what are the differences between the legacy U-Boot WTMI-armada-17.10.5-bb8f823 and the one used by Armbian ( WTMI-devel-18.12.1-e6bb176 )? The unmodified boards boot perfectly each time, but when I change to Armbian u-boot, no matter what frequency (I tried 800 and 1000 MHz), the boards need one to four power cycles till they boot even into u-boot. On serial console, I get:

TIM-1.0
WTMI-devel-18.12.1-e6bb176
WTMI: system early-init
SVC REV: 5, CPU VDD voltage: 1.108V

And that's it. I need to pull the power plug and put it back in for several times, then finally the board boots fine (and runs fine ever after). The voltage varies depending on the frequency set by loading different u-boot images.

 

Has anyone a clue?

[martinayotte]

1 hour ago, barish said:

As a workaround, try putting the following line into /boot/armbianEnv.txt:

rootdev="/dev/mmcblk1p1"

That is only for specifying U-Boot location and passing it to kernel ... But :

 

The "ALERT! /dev/mmcblk0p1 does not exist. Dropping to a shell!" really means that /etc/fstab isn't trying to mount proper partition.

[barish]

3 hours ago, martinayotte said:

That is only for specifying U-Boot location and passing it to kernel ... But :

 

The "ALERT! /dev/mmcblk0p1 does not exist. Dropping to a shell!" really means that /etc/fstab isn't trying to mount proper partition.

Then change fstab device to mmcblk1p1 or even better to the UUID of the SD card, no?

[barish]

4 hours ago, barish said:

Can anyone tell me what are the differences between the legacy U-Boot WTMI-armada-17.10.5-bb8f823 and the one used by Armbian ( WTMI-devel-18.12.1-e6bb176 )? The unmodified boards boot perfectly each time, but when I change to Armbian u-boot, no matter what frequency (I tried 800 and 1000 MHz), the boards need one to four power cycles till they boot even into u-boot.

By now, I found some more information regarding TIM and WTMI. According to Marvell's A3700-utils documents, TIM is a first aproach to address DDR3/DD4 at fixed freq of 300/400MHz, while WTMI "performs dynamic DDR PHY training" to gain higher memory speed. Apparently, the Armbian U-Boot uses a later version of WTMI (and maybe uses different parameters?), does anyone know about this?

[barish]

I changed now to the legacy U-Boot (17.10) because it boots very reliably (which is what I need). I am not sure of the negative effects (testing is ongoing) but there is no option for me at the moment. I hope that someone with U-Boot and/or Marvell skills can tell me what changes from 17.10 -> 18.12 led to the unreliable booting. Since it takes place at the start of the WTMI RAM tuning process, I can only assume that there is the rub.

[Andrius]

Thanks for you advices.

First I checked /etc/fstab and found that it already contains the entry with the correct UUID:

UUID=a02e20f8-402e-4469-87e0-9d0c9507ea15 / ext4 defaults,noatime,nodiratime,commit=600,errors=remount-ro 0 1

Then, as suggested, I updated /boot/armbianEnv.txt with rootdev="/dev/mmcblk1p1". 

Strange thing was that this file contained some apparently zero chars at the end which did not make happy some text editors:

 

rootdev="/dev/mmcblk1p1"

/var/log.hdd/apt/term.log {
  rotate 12
  monthly
  compress
  missingok
  notifempty
}

/var/log.hdd/apt/history.log {
  rotate 12
  monthly
  compress
  missingok
  notifempty
}

^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@

I let it boot and got somewhat different output, but still no go:

 

Starting kernel ...

[    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd034]
[    0.000000] Linux version 4.19.56-mvebu64 (root@armbian.com) (gcc version 7.4.1 20181213 [linaro-7.4-2019.02 revision 56ec6f6b99cc167ff0c2f8e1a2eed33b1edc85d4] (Linaro GCC 7.4-2019.02)) #5.89 SMP PREEMPT Tue Jun 25 22:27:47 CEST 2019
[    0.000000] Machine model: Globalscale Marvell ESPRESSOBin Board
[    0.000000] earlycon: ar3700_uart0 at MMIO 0x00000000d0012000 (options '')
[    0.000000] bootconsole [ar3700_uart0] enabled
Loading, please wait...
starting version 237
Begin: Loading essential drivers ... done.
Begin: Running /scripts/init-premount ... done.
Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done.
Begin: Running /scripts/local-premount ... Scanning for Btrfs filesystems
done.
Begin: Will now check root file system ... fsck from util-linux 2.31.1
Checking all file systems.
done.
mount: mounting "/dev/mmcblk1p1" on /root failed: No such file or directory
done.
Begin: Running /scripts/local-bottom ... done.
Begin: Running /scripts/init-bottom ... mount: mounting /dev on /root/dev failed: No such file or directory
mount: mounting /dev on /root/dev failed: No such file or directory
done.
mount: mounting /run on /root/run failed: No such file or directory
run-init: current directory on the same filesystem as the root: error 0
Target filesystem doesn't have requested /sbin/init.
run-init: current directory on the same filesystem as the root: error 0
run-init: current directory on the same filesystem as the root: error 0
run-init: current directory on the same filesystem as the root: error 0
run-init: current directory on the same filesystem as the root: error 0
run-init: current directory on the same filesystem as the root: error 0
No init found. Try passing init= bootarg.


BusyBox v1.27.2 (Ubuntu 1:1.27.2-2ubuntu3.2) built-in shell (ash)
Enter 'help' for a list of built-in commands.

(initramfs)

Should I remove those pesky zeros? Are they a sign the file system has problems?

 

[Andrius]

Nope. Zeros or no zeros - same problems.

[barish]

10 hours ago, Andrius said:

Nope. Zeros or no zeros - same problems.

My guess is that your Linux got corrupted and you should copy the backup of your SD card to a new SD card and try with that.

[bigbrovar]

On 5/7/2020 at 11:01 PM, barish said:

Despite of all the seemingly bad news, I boldly ordered a batch of 10 EspressoBin V7 recently. Globalscale promised to test them for me for stability (I am not interested in squeezing out the last bit of performance), and I tested them again after reception: They all seem to run stable with Ubuntu Xenial (the "newest" image provided by Globalscale) and with Armbian kernel 4.19.59-mvebu64.

Since there is still a few weeks till they go on duty, if anyone is interested in a particular test on them, I am willing to do so. I am not into kernel development, so I would need some instruction. ;-)

Hey, I am considering getting this board as a openwrt router. How where you able to get them to commit to testing it before purchasing. Also have you had any issues since?

[barish]

6 hours ago, bigbrovar said:

Hey, I am considering getting this board as a openwrt router. How where you able to get them to commit to testing it before purchasing. Also have you had any issues since?

I purchased 10 and will most probably buy more in future. :-P I had no issues with U-Boot 17.10 despite intensive testing. If you are in Germany, I can offer you a tested EspressoBin v7 1GB.

[bigbrovar]

On 6/24/2020 at 1:33 PM, barish said:

I purchased 10 and will most probably buy more in future. :-P I had no issues with U-Boot 17.10 despite intensive testing. If you are in Germany, I can offer you a tested EspressoBin v7 1GB.

I think I might have to take you up on this offer. I don't stay in Germany but I am sure I can be shipped to.

[bigbrovar]

On 6/24/2020 at 1:33 PM, barish said:

I purchased 10 and will most probably buy more in future. :-P I had no issues with U-Boot 17.10 despite intensive testing. If you are in Germany, I can offer you a tested EspressoBin v7 1GB.

Can you send me a message or how do we make this happen?

[FlashBurn]

Is there a time frame when espressobin will also have a newer kernel version? I tried arch linux with kernel 5.8 and there the espressobin was working as intended, but on the armbian kernel 4.19.120 (or something like that) it is still not running with the right frequency

[lanefu]

hey @FlashBurn

v20.08 armbian will have kernel 5.6. you can test it out on a nightly image now if you like

[FlashBurn]

4 hours ago, lanefu said:

hey @FlashBurn

v20.08 armbian will have kernel 5.6. you can test it out on a nightly image now if you like

How? I wanted to change the kernel, but there are only old kernels on armbian-config. So how can I change to a nightly built? Or do you mean from the download site? This hasn´t worked for me yesterday, but I can try tomorrow again.

 

Edit::

Nope, still not working (nightly built download). Goes right to 404

Edited August 17, 2020 by FlashBurn

[lanefu]

DM'd you an image link