Review Quick review of Solidrun's Clearfog

[Igor]

So whats missing in mainline?

 

Basic things (what I have tested) are working but we need to do more tests. I think fiber is not done yet since there some patches left behind and need to be fixed. Will do that in next update. Stability in 4.4.x needs to be tested ...

 

It can be a router, NAS and also SOHO application server.

[Toast]

tnx Igor will be monitoring progress for the Clearfog since it is a interesting hardware lots of potential 

[Pontus Jon Jensen Karlsson]

Following this thread with great interest, I'm myself looking into perhaps buying this board.

I haven't quite decided though between the ClearFog Pro or a Turris Omnia, their specs are really similar (Turris able to ship with 2GB of RAM though!).

 

However I'm a bit puzzled by ClearFog's specification stating that they are able to ship with either A380 or A388, where the A388 comes with 1GB of RAM as well as 4GB of eMMC.

When I go to order it I never get an option between both versions. So which specifications is it?

[Igor]

I am meeting board designer tomorrow ... I am going to ask/check up those things. My test board is suppose to be the most powerful but it does not have eMMC. I also don't see much need since it's simple and fast enough to load u-boot from MMC, the rest anyway can go from M2, sata or usb.

[Pontus Jon Jensen Karlsson]

That would be great, here's a quote from the Specification page:

ARMADA based A380/A388
256MB (A380)/1GB (A388)
M.2
8GB uSD/4GB eMMC (Optional)

 

Now that I re-read it I realise it actually says 4GB eMMC is optional, so that's my bad but I still didn't get an option for this optional feature.

 

I had a look at the spec. page for their ARMADA SoM family as well and the A380 SoM hasn't been released yet, so my guess is that it's actually the A388 they ship.

[Igor]

As far as I know they stock one version at the moment, without modifications. Clearfog PRO, the one which is in our test. I also could be wrong ... More info in few days.

[Pontus Jon Jensen Karlsson]

Do you know if it's possible to boot the board from M.2 or does U-boot have to be located on the microSD?

[Igor]

Don't know, presumably not. would need to check.

[Igor]

Do you know if it's possible to boot the board from M.2 or does U-boot have to be located on the microSD?

 

I got confirmation that it's possible to boot from M2 with modified u-boot.

[Pontus Jon Jensen Karlsson]

Terrific! I got a reply from SolidRun on an inquire about the optional eMMC. It is possible to order through their support, it's an additional $7 which means a total of $177 for the board with eMMC and the uSOM module.

 

The uSOM module delivered with the ClearFog Pro is indeed the A388.

[Pontus Jon Jensen Karlsson]

So I received my board today, although been unable to boot it up.

I've tried with three different SD cards and two different images, latest armbian as well as latest ArchLinuxARM.

Regardless what I do the only output I'm given through the FTDI console is;

BootROM - 1.73

Booting from MMC
BootROM: Bad header at offset 00000000
BootROM: Bad header at offset 00200000
Switching BootPartitions.
BootROM: Bad header at offset 00000000
BootROM: Bad header at offset 00200000
Switching BootPartitions.
BootROM: Bad header at offset 00000000
BootROM: Bad header at offset 00200000
Switching BootPartitions.
BootROM: Bad header at offset 00000000
BootROM: Bad header at offset 00200000
Switching BootPartitions.
BootROM: Bad header at offset 00000000
BootROM: Bad header at offset 00200000
Switching BootPartitions.
BootROM: Bad header at offset 00000000
BootROM: Bad header at offset 00200000
Switching BootPartitions.
BootROM: Bad header at offset 00000000
BootROM: Bad header at offset 00200000
Switching BootPartitions.
BootROM: Bad header at offset 00000000
BootROM: Bad header at offset 00200000
Switching BootPartitions.
BootROM: Bad header at offset 00000000
BootROM: Bad header at offset 00200000
Switching BootPartitions.
BootROM: Bad header at offset 00000000
BootROM: Bad h
Trying Uart

I've contacted the Support for more details, perhaps it's something I'm missing due to having the eMMC addition.

Have you seen it before during your attempts?

[Igor]

Usually when SD card is inserted it overrides eMMC. It's possible that u-boot needs some adjustments when eMMC is present.

 

Let's wait for Solidrun answer on this.

[Pontus Jon Jensen Karlsson]

What output are you given when you attempt to boot the board without an SD card inserted?

 

I'm not clear on what stage is actually attempted to be loaded at offset 00000000, but that should be the BootROM responsible for loading U-boot,

so it appears it's actually whatever software that needs to load u-boot is broken?

 

It seems though that eMMC disconnects SD IO; 

 

 Embedded flash. This option disconnects SDIO from Board-to-Board (see page-2)

quoted from http://wiki.solid-run.com/lib/exe/fetch.php?media=a38x:microsom:docs:a38x-microsom-schematics-simplified-rev2.00.pdf.

 

However would I be expected to load anything to the eMMC without any way to actually boot the system?

[Igor]

Without MMC:

Booting from MMC
Card did not respond to voltage select!
Error initializing MMC - FFFFFFEF


Trying Uart 

With MMC:

BootROM - 1.73

Booting from MMC


General initialization - Version: 1.0.0
Detected Device ID 6828
High speed PHY - Version: 2.0

Init Customer board board SerDes lanes topology details:
 | Lane # | Speed|    Type     |
 ------------------------------|
 |   0    |  3   |  SATA0      |
 |   1    |  0   |  SGMII1     |
 |   2    |  5   |  PCIe1      |
 |   3    |  5   |  USB3 HOST1 |
 |   4    |  5   |  PCIe2      |
 |   5    |  0   |  SGMII2     |
 -------------------------------
:** Link is Gen1, check the EP capability 
PCIe, Idx 1: remains Gen1
PCIe, Idx 2: detected no link
High speed PHY - Ended Successfully
DDR3 Training Sequence - Ver TIP-1.39.0
DDR3 Training Sequence - Switching XBAR Window to FastPath Window 
DDR3 Training Sequence - Ended Successfully
BootROM: Image checksum verification PASSED

 __   __                      _ _
|  \/  | __ _ _ ____   _____| | |
| |\/| |/ _` | '__\ \ / / _ \ | |
| |  | | (_| | |   \ V /  __/ | |
|_|  |_|\__,_|_|    \_/ \___|_|_|
         _   _     ____              _
        | | | |   | __ )  ___   ___ | |_ 
        | | | |___|  _ \ / _ \ / _ \| __| 
        | |_| |___| |_) | (_) | (_) | |_ 
         \___/    |____/ \___/ \___/ \__| 
 ** LOADER **


U-Boot 2013.01 (Feb 18 2016 - 09:36:35) Marvell version: 2015_T1.0p11

[Pontus Jon Jensen Karlsson]

Solid-run still hasn't gotten back to me on this issue but I've traced the points myself and realised that SD/SDIO/MMC shares the same lines as eMMC,

so yes getting the eMMC effectively disabled the SD card slot.

 

I will urge them to document this on the wiki though, since it's not really clear unless you follow the schematics.

 

So I guess I'll try and boot U-boot from UART, then PXE boot the kernel, then install the base system!

[Igor]

So I guess I'll try and boot U-boot from UART, then PXE boot the kernel, then install the base system!

 

I think that's actually the only option. A bit more complicated.

[Pontus Jon Jensen Karlsson]

Actually, there's the option to flash through JTAG as well I think?

But Marvell officially only supports Lauterbach and their own JTAG debuggers, neither of them goes for any less than $300.

 

So while there's other alternatives, this is the cheapest and the only one I'm able to try with the resources I have at home right now.

[Quintus23M]

@Igor,
in the meantime, I was able to run some first tests with the ClearFog Pro:
 

1) Using a fast mSATA SSD
 

Samsung SSD 840 EVO mSATA 500GB

 

Write:

1048576000 bytes (1.0 GB) copied, 3.06723 s, 342 MB/s (write cache=ON)
1048576000 bytes (1.0 GB) copied, 8.91889 s, 118 MB/s (write cache=OFF)

Read:

1048576000 bytes (1.0 GB) copied, 1.96029 s, 535 MB/s

 

2) Ethernet connection over SFP

Connecting two ClearFog Pro over a SFP fiber link. 

# ethtool eth1
Settings for eth1:
        Supported ports: [ FIBRE ]
        Supported link modes:   1000baseT/Full
        Supported pause frame use: Symmetric
        Supports auto-negotiation: Yes
        Advertised link modes:  1000baseT/Full
        Advertised pause frame use: Symmetric
        Advertised auto-negotiation: Yes
        Speed: 1000Mb/s
        Duplex: Full
        Port: FIBRE
        PHYAD: 0
        Transceiver: external
        Auto-negotiation: on
        Link detected: yes

Network speed was just 1Gbit/sec, we were not able to change the settings to get up to 2.5Gbit/sec.

[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec  1.10 GBytes   942 Mbits/sec    0             sender
[  4]   0.00-10.00  sec  1.10 GBytes   941 Mbits/sec                  receiver

We used the following GBIC's "Brocade GBIC 4 Gbit/s SW FC SFP - 57-1000013-01".

Any clues how to get the maximum ethernet speed over SFP? Maybe for 2.5G there has to be some special settings, or there is something missing in the kernel drivers to support 2.5G. 

Update: 

The 2.5Gbit mode for SFP can be enabled with a patch in U-Boot. I already have prepared the patch and compiled the new u-boot, but I'm having trouble to boot the new image because u-boot has some problems. (will update this post as soon as I have a solution)
 

Update2: (13.03.2016)

We were able to successfully test the 2.5Gbit mode for SFP, using two ClearFog Pro boards connected directly with two Brocade GBIC's and a 1m optical fibre cable. Testing with `iperf3` shows us the maximum network speed which is really excellent.

[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec  2.74 GBytes  2.35 Gbits/sec    0             sender
[  4]   0.00-10.00  sec  2.74 GBytes  2.35 Gbits/sec                  receiver

3) COMPEX WLE900VX-7A miniPCIe module

This WiFi module has just arrived, hope to get it running soon...
Specs: AR9880, 802.11ac, 2,4/5GHz, 3*3 MIMO

Update:

WiFi is working, I just added the correct kernel modules and recompiled and installed the new kernel 4.4.4.

CONFIG_ATH10K=m
CONFIG_ATH10K_PCI=m

Additionally I had to install the correct firmware files for the Compex WLE900VX card:

/lib/firmware/ath10k/QCA988X/hw2.0/firmware-5.bin_10.2.4.70.22-2
/lib/firmware/ath10k/QCA988X/hw2.0/firmware-5.bin -> firmware-5.bin_10.2.4.70.22-2
/lib/firmware/ath10k/QCA988X/hw2.0/board.bin

[solarnetone]

I notice nobody mentioning the 88e6176.  That as well required a 4.4.4 kernel. 

 

CONFIG_HAVE_NET_DSA=y
CONFIG_NET_DSA=y
CONFIG_NET_DSA_HWMON=y
CONFIG_NET_DSA_TAG_EDSA=y
CONFIG_NET_DSA_MV88E6XXX=m
CONFIG_NET_DSA_MV88E6352=m
 

Simply add mv88e6xxx_drv to /etc/modules after compilation.  Interfaces are presented as lan1-lan6, such that normal network control tools like ip, ifconfig, and properly configured ifup/ifdown work as expected.

[whatever4711]

I notice nobody mentioning the 88e6176.  That as well required a 4.4.4 kernel. 

 

CONFIG_HAVE_NET_DSA=y

CONFIG_NET_DSA=y

CONFIG_NET_DSA_HWMON=y

CONFIG_NET_DSA_TAG_EDSA=y

CONFIG_NET_DSA_MV88E6XXX=m

CONFIG_NET_DSA_MV88E6352=m

 

Simply add mv88e6xxx_drv to /etc/modules after compilation.  Interfaces are presented as lan1-lan6, such that normal network control tools like ip, ifconfig, and properly configured ifup/ifdown work as expected.

 

Thanks for the Kernel parameter. It works nearly perfect, but I am missing one interface:

lan1@eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noqueue state DOWN group default qlen 1000    link/ether 00:50:43:84:25:2f brd ff:ff:ff:ff:ff:ff
lan2@eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noqueue state DOWN group default qlen 1000    link/ether 00:50:43:84:25:2f brd ff:ff:ff:ff:ff:ff
lan3@eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noqueue state DOWN group default qlen 1000    link/ether 00:50:43:84:25:2f brd ff:ff:ff:ff:ff:ff
lan4@eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noqueue state DOWN group default qlen 1000    link/ether 00:50:43:84:25:2f brd ff:ff:ff:ff:ff:ff
lan5@eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noqueue state DOWN group default qlen 1000    link/ether 00:50:43:84:25:2f brd ff:ff:ff:ff:ff:ff

During bootstrap, it shows:

dsa dsa@0 lan6 (uninitialized): no phy at 6
dsa dsa@0 lan6 (uninitialized): failed to connect to port 6: -19
mvneta f1030000.ethernet eth0: error -19 setting up slave phy
mvneta f1030000.ethernet eth0: [0]: can't create dsa slave device for port 6(lan6): -19

Current configuration: Kernel 4.4.5 with Quintus' 2.5Gbit patch for the SFP.

 

Accessible ports:

 

[Pontus Jon Jensen Karlsson]

If anybody is following this thread awaiting a solution for booting a eMMC board; I'm still empty handed.

Truth is since this isn't my daytime job I have very limited time to try things out, even less these past few weeks.

 

I did get last night off and tried some stuff but regardless my attempts I was empty handed.

I'm now trying to compile the u-boot image myself rather than patching existing ones, but the u-boot fork 

doesn't seem to have support for booting over UART.

 

I did find something slightly amusing though, there seems to be two u-boot forks implementing the a38x support.

There's the MarvellEmbeddedProcessors which SolidRun has forked of, then there's the mainline support now based

on code from Stefan Roese (https://www.mail-archive.com/u-boot@lists.denx.de/msg189083.html).

 

The Turris Omnia project has a u-boot fork of the mainline version, which also has fixes for the UART kwb image generation.

Since the Turris Omnia will be shipping in just a month I assume their fork is somewhat stable so I will harvest whatever

of the UART patches I can and see if any of it fixes my problems.

 

Funny thing is though that SolidRun tech. support guys boot through UART, but they never replied on my last email so

I'm clueless on how they do it.

 

Edit: Finally got the damn thing booting from UART, it all boiled down to the timing of the first xmodem package basically.

What I did was I put the kwboot command line in a while loop and reset the device every time I got the "Bad message".

After about ten attempts it finally booted! 

[solarnetone]

Thanks for the Kernel parameter. It works nearly perfect, but I am missing one interface:

lan1@eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noqueue state DOWN group default qlen 1000    link/ether 00:50:43:84:25:2f brd ff:ff:ff:ff:ff:ff
lan2@eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noqueue state DOWN group default qlen 1000    link/ether 00:50:43:84:25:2f brd ff:ff:ff:ff:ff:ff
lan3@eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noqueue state DOWN group default qlen 1000    link/ether 00:50:43:84:25:2f brd ff:ff:ff:ff:ff:ff
lan4@eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noqueue state DOWN group default qlen 1000    link/ether 00:50:43:84:25:2f brd ff:ff:ff:ff:ff:ff
lan5@eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noqueue state DOWN group default qlen 1000    link/ether 00:50:43:84:25:2f brd ff:ff:ff:ff:ff:ff

During bootstrap, it shows:

dsa dsa@0 lan6 (uninitialized): no phy at 6
dsa dsa@0 lan6 (uninitialized): failed to connect to port 6: -19
mvneta f1030000.ethernet eth0: error -19 setting up slave phy
mvneta f1030000.ethernet eth0: [0]: can't create dsa slave device for port 6(lan6): -19

Current configuration: Kernel 4.4.5 with Quintus' 2.5Gbit patch for the SFP.

 

Sounds like either a)  said patch is allocating a PHY for the fiber interface, or otherwise interfering with the phy for lan6 OR, some other factor, like lack of kernel module, or physical defect, is preventing the PHY from coming up.  What does dmesg|grep phy say?

[Guest PA]

@Pontus,

 

How did you set up u-boot? I have a clearfog board with emmc, that I only get partially through with kwboot.

I have set the "BOOT_FROM" parameter to uart.

Leaving the dip-switches in "SD / MMC mode" it tries to load the next step from MMC1, setting it to UART I get the following error:

 

 Trying to boot from unknown boot device
SPL: Unsupported Boot Device!
SPL: failed to boot from all boot devices
 

I have compiled mainline u-boot with the clearfog_config, can you share your setup?

 

[xueliu]

Hello everyone,

 

Could someone can give me a working config file for 4.4 or 4.5, and tell me which kernel tree do you use. Thanks a lot.

 

Regards,

 

Xue Liu

[rabeeh]

Solid-run still hasn't gotten back to me on this issue but I've traced the points myself and realised that SD/SDIO/MMC shares the same lines as eMMC,

so yes getting the eMMC effectively disabled the SD card slot.

 

I will urge them to document this on the wiki though, since it's not really clear unless you follow the schematics.

 

So I guess I'll try and boot U-boot from UART, then PXE boot the kernel, then install the base system!

 

Just updated the wiki about booting from uart / micro sd / eMMC

 

Notice the script to automate loading u-boot-uart.mmc

 

This uses SolidRun uart and not the upstream one.

http://wiki.solid-run.com/doku.php?id=products:a38x:software:development:u-boot

[Guest solarnetone]

Hi all,

 

A few questions and observations: 

I snagged a nice little transcend m.2 ssd.  After partitioning and rsyncing the root filesystem, I want to be able to boot from this drive.  To this end, I plan to compile uboot, and dd the resultant binary to /dev/sda (the m.2) with block size 512. 

Question 1:  Do I need to offset a block, or not? 

Question 2:  Are there any config or compile time options, or patches that need be applied to the marvell uboot tree in order to successfully boot from ssd? 

Question 3:  Same question as #2, as respects mainline u-boot? 

Question 4:  Need I fool with the DIP switches in order to boot from other than SD/EMMC? 

Question 5:  What is the function of those switches? 

 

Assuming I can reach a u-boot shell when booting from the m.2, it should be not much more that building a new boot.scr, and editing fstab, correct?

 

Regarding the SPF interface, I also acquired a Brocade 57-1000013-01.  Can I have a pointer to the relevant patch?

 

Thanks,

Scott

[Pontus Jon Jensen Karlsson]

Just updated the wiki about booting from uart / micro sd / eMMC

 

Notice the script to automate loading u-boot-uart.mmc

 

This uses SolidRun uart and not the upstream one.

http://wiki.solid-run.com/doku.php?id=products:a38x:software:development:u-boot

 

I've been away from this project of mine for a month now and was very happy to see this!

 

So in order to get this thing booting and cooperative I built a toolchain with crosstool-ng and built u-boot.

download-serial.sh was acting a bit wonky at first so I had to give it a few tries in order to get the timing right but it did boot.

 

After this I loaded the u-boot.mmc from over serial with xmodem into memory and wrote it, here's where it got a bit tricky though.

It refuses to boot from mmc dev 0 1, I had to write to partition 0 for it to actually boot from eMMC.

 

Thank you very much for taking the time and write this information, it was extremely helpful!

[jetebony]

I have just downloaded the 4.6.2 kernel version of xenial and get the lan1 to lan6 interfaces appear, but I can not seem to configure them.

 

As mentioned previously, in /var/log/kern.log i see this:

 

mvneta f1030000.ethernet eth1: [0]: couldn't create dsa switch instance (error -22)

 

Which i guess is part of the problem. Are there any clues as to what I need to do to get these interfaces configured/working? According to the link in the in the post below:

 

"Interfaces are presented as lan1-lan6, such that normal network control tools like ip, ifconfig, and properly configured ifup/ifdown work as expected"

 

But, I can not seem to do anything with these interfaces lan1 to lan6?

 

Any pointers would be very helpful.

 

Cheers,

Nigel.

[Guest oli]

Is there some guide in howto compile uboot? i tried to follow the tutorial at http://wiki.solid-run.com/doku.php?id=products:a38x:software:development:u-bootbut it fails:

[oli@localhost u-boot-armada38x]$ export CROSS_COMPILE=arm-linux-gnueabi-
[oli@localhost u-boot-armada38x]$ make armada_38x_clearfog_config
Configuring for armada_38x_clearfog - Board: armada_38x, Options: CUSTOMER_BOARD_1,ARMADA_38X,MV_MMC_BOOT,MV_INCLUDE_SPI,DDR3
[oli@localhost u-boot-armada38x]$ make u-boot.mmc
Generating include/autoconf.mk
In file included from /home/oli/u-boot-armada38x/include/linux/compiler.h:40:0,
                 from /home/oli/u-boot-armada38x/include/part_efi.h:32,
                 from /home/oli/u-boot-armada38x/include/part.h:188,
                 from include/common.h:116:
/home/oli/u-boot-armada38x/include/linux/compiler-gcc.h:87:30: fatal error: linux/compiler-gcc6.h: No such file or directory
 #include gcc_header(__GNUC__)
                              ^
compilation terminated.
Generating include/autoconf.mk.dep
In file included from /home/oli/u-boot-armada38x/include/linux/compiler.h:40:0,
                 from /home/oli/u-boot-armada38x/include/part_efi.h:32,
                 from /home/oli/u-boot-armada38x/include/part.h:188,
                 from include/common.h:116:
/home/oli/u-boot-armada38x/include/linux/compiler-gcc.h:87:30: fatal error: linux/compiler-gcc6.h: No such file or directory
 #include gcc_header(__GNUC__)
                              ^
compilation terminated.

arm-linux-gnueabi-gcc -DDO_DEPS_ONLY \
	-g  -Os   -fno-common -ffixed-r8 -msoft-float  -D__KERNEL__ -I/home/oli/u-boot-armada38x/include -fno-builtin -ffreestanding -nostdinc -isystem /usr/lib/gcc/arm-linux-gnueabi/6.1.0/include -pipe  -DCONFIG_ARM -D__ARM__ -marm -mno-thumb-interwork -mabi=aapcs-linux -march=armv7-a -march=armv7-a  -Wall -Wstrict-prototypes -fno-stack-protector -Wno-format-nonliteral -Wno-format-security -fstack-usage   \
	-o lib/asm-offsets.s lib/asm-offsets.c -c -S
In file included from /home/oli/u-boot-armada38x/include/linux/compiler.h:40:0,
                 from /home/oli/u-boot-armada38x/include/part_efi.h:32,
                 from /home/oli/u-boot-armada38x/include/part.h:188,
                 from /home/oli/u-boot-armada38x/include/common.h:116,
                 from lib/asm-offsets.c:18:
/home/oli/u-boot-armada38x/include/linux/compiler-gcc.h:87:30: fatal error: linux/compiler-gcc6.h: No such file or directory
 #include gcc_header(__GNUC__)
                              ^
compilation terminated.
Makefile:725: die Regel für Ziel „lib/asm-offsets.s“ scheiterte
make: *** [lib/asm-offsets.s] Fehler 1

Seems like there is only gcc3 and gcc4 supported?

[oli@localhost u-boot-armada38x]$ find . -name compiler*
./include/compiler.h
./include/linux/compiler.h
./include/linux/compiler-gcc4.h
./include/linux/compiler-gcc.h
./include/linux/compiler-gcc3.h

I would love to boot from the mSATA (Transcend 32gb)

[Guest oli]

Guess i will try it today with igor's compile.sh - looks nice. But a question regarding the patches to enable/disable mpcie/msata - can someone tell me if i have to rename any of the patches from .disabled to .enabled so they get used?