Jump to content


  • Posts

  • Joined

Posts posted by tkaiser

  1. On 12/12/2018 at 2:04 PM, kcn said:

    I manualy uploaded some armbianmonitor logs: https://pastebin.com/VGrMddE5

    ### boot environment:
    #   $OpenBSD: sshd_config,v 1.100 2016/08/15 12:32:04 naddy Exp $
    # This is the sshd server system-wide configuration file.  See
    # sshd_config(5) for more information.
    # This sshd was compiled with PAT

    Smells like filesystem corruption. If the contents of /boot/armbianEnv.txt really look like this garbage you should immediately check installation integrity using armbianmonitor -v.

  2. On 9/29/2018 at 5:48 AM, Jason Law said:

    On a different note regarding the heat sink, are you better off with a fan blowing from the top down? Or from the side for heat dissipation?


    Depends. Though in all my tests blowing air laterally performed best (see here or here). With NanoPi M4 just some airflow through the heatsink fins is already sufficient (tested it behind a server PSU's 'hot air' outlet).


    But of course you can easily conduct your own tests. Simply use sbc-bench's thermal testing mode and if you want to adjust your fan setup interactively then use the installed cpuminer application in /usr/local/src/ (but please keep in mind that with the huge thermal mass of NanoPi M4's heatsink it's very easy too fool yourself and that's why I would always recommend a combined sbc-bench -t/-T test run)


    1 hour ago, Teal said:

    If it would be a Marvell based 4-port SATA - and omv compatible like this one for example

    I recommended the 88SE9235 for a reason and @mindee said they would use this chip. I hope you're aware that there exist RK3399 boards with full PCIe slots already? PCIe functionality is only a kernel thing (drivers and device-tree settings) and a board design issue (PCB trace lengths/routing and signal quality)

  3. 10 hours ago, sfx2000 said:

    Interesting numbers


    100% meaningless numbers now since they do not provide any insights any more (you can not differentiate what's throttling and what's -- potentially inappropriate -- cpufreq scaling behavior).


    10 hours ago, sfx2000 said:

    RK3288 can do the limbo as low as 126MHz - right now with Armbian we're capped at the bottom at 600MHz


    For two simple reasons:

    • allowing very low cpufreq OPP trashes performance (on your Tinkerboard it's storage performance due to the clockspeeds not increasing fast enough)
    • the way DVFS works at the lower end of the scale the differences in consumption and generated heat are negligible between silly ultra low MHz values and a reasonable lower limit (we in Armbian TEST for stuff like this with every new SoC family!)

    Closed your issue right now spending additional time to explain: https://github.com/ThomasKaiser/sbc-bench/issues/4


  4. 5 hours ago, sfx2000 said:

    why mount the zram partion as ext4-journaled


    Read the script and don't just skim through. This script (initially written by Igor and not me -- there's also a commit history which helps you understand what stuff is dealt with here) sets up things that need to work on any of the kernels Armbian supports (we're not just dealing with the few devices you seem to use) and does a couple of different stuff. Of course we're not using filesystems for swap and you might even look into the Changelog to understand why mkfs.ext4 is used there..


    @Igor certificate for docs.armbian.com has expired

  5. 4 minutes ago, chwe said:

    maybe we should add 'dumb mode' to known issues for those boards?


    Sure, feel free to create an 'USB-C powered boards -- important information' thread or something like that.


    5 minutes ago, chwe said:
    3 hours ago, tkaiser said:

    You're talking about some Apple smartphones and tablets still using their Lightning connector and not USB-C?

    If those cables were not 30$ and the cheap fakes destroyed the phone. 


    How is that different with USB-C? See https://www.makeuseof.com/tag/best-usb-c-chargers-whats-safe-whats-dangerous/ or https://www.theverge.com/2016/2/4/10916264/usb-c-russian-roulette-power-cords for example.

  6. 8 hours ago, sfx2000 said:

    just wait for USB-C connectors on SBC's


    They're there already: Firefly RK3399, Vim, Vim2, RockPro64, NanoPC-T4, NanoPi M4, Ficus, Renegade Elite and soon also NanoPi NEO4, Khadas Edge and Edge-V are equipped with USB-C.


    Some of these boards use USB-C as power input. Here we need to differentiate between 'dumb mode' and 'USB PD compliant' (USB PD --> USB Power delivery specs):

    So as long as there are not real USB-C chargers lying around in every household and all SBC are USB PD compliant USB-C for powering SBC is simply either a mess or an expensive adventure.


    8 hours ago, sfx2000 said:

    (perhaps the reason why the folks over on Cupertino decided with their recent HW releases to keep their proprietary connection)


    You're talking about some Apple smartphones and tablets still using their Lightning connector and not USB-C?


    Most probably related to the count of peripherals that already use Lightning (Docking Stations and stuff like that). Apple was the company pushing hard for USB market adoption 20 years ago (giving up on their own proprietary ADB bus standard which resulted in a lot of users like me having to buy adapters for expensive ADB peripherals like graphics tablets and such stuff) and they also sent truckloads of engineers to USB Implementers Forum years ago when USB-C and USB PD were defined. Most probably to prevent USB-IF ever doing something silly as Micro USB again (so that they had to invent something own as Lightning since Micro USB already sucked back then).

  7. 14 minutes ago, codnoscope said:

    hikey960 for example, its NVMe slot is orientated outward


    Isn't it the same with the normal Rock960? To me personally not a single 96boards specs compliant device looked appealing (unfortunately Ficus included). But it's the 'standard' itself that's the problem. Form factor, placement of connectors, interfaces exposed.


    The PCIe connector placement on EE boards is simply moronic. But maybe I understood the whole 96boards approach wrong and this was never meant for end users but just for Android developers?

  8. Almost forgot: in case someone wants to add this device to Armbian as CSC (community supported config -- for official support a 'Board Bring-Up' thread/discussion is needed). I would believe starting with a ficus.csc like this would be a good idea:

    # RK3399 hexa core 1-4GB GBE eMMC USB3 USB-C WiFi PCIe

    Then adding the 4.4 DT as u-boot and default kernel patch, the mainline DT as dev kernel patch and loading the specific DT similar to how we do it already. Open question: how to deal with rk3399_loader_v1.12.112.bin?

  9. 18 hours ago, codnoscope said:

    Don't get too excited about the BMC, the board itself do not have one, you need their daughter board on the M2 slot to act as an out of band management controller


    That's how Tom described it half a year ago. Hopefully we get some more details soon. But all this 'BMC' seems to provide is serial console access and remote power cycling (something I use dirt cheap SBC with USB hubs and USB-UART adapters for combined with my two USB controlled EnerGenie power outlets). Standard in x86 world in the meantime is full graphical console through BMC (usually using an ASpeed controller that appears as a PCIe GPU to the host).


    I'm a believer into 'use case first' and have still a hard time finding such a use case for this board. If the pricing back from April is still valid the board with just 1 GB RAM might be available for $99. An RK3399 thingy also with just 1 GB DRAM (though single channel config so lower memory bandwidth) might be available soon for less than 60 bucks (NanoPi NEO4). So why choosing the Ficus then?

    • Compatible to 96boards Mezzanines? Nope, I don't need any of them (the PoE thingy looked interesting until I found out that here Gigabit Ethernet is attached via USB2 and they use the same crappy chip as on the new RPi 3 B+ which is a total fail for more than one reason)
    • 2 SATA ports? Well, performance of the JMS561 implementation is ok for HDDs but there's the SMART issue (and for me as a storage professional disks that can't be monitored can not be used). And then 2 disks need powering (probably solved here since there's DC-DC circuitry on the board and 2 SATA power ports with both 5V and 12V. But this setup with 2 x 3.5" HDD and just a 12V/4A PSU might be asking for troubles due to HDD spin-up peak consumption requirements). And 2 disks also need housing. So where's an enclosure appropriate to fit the board and 2 disks (either 2.5" or 3.5")?
    • But there's a full size PCIe slot? Sure but if my use case requires PCIe then there's a way cheaper RK3399 alternative (RockPro64) and honestly the board layout sucks (96board EE specs). Any PCIe card when inserted into this slot protrudes 25mm over the board. And with a PCIe card I also want something like an enclosure to fixate the card and protect the whole setup. But with the weird PCIe slot placement and the proprietary form factor there aren't inexpensive enclosures available. This 96boards EE form factor is both too huge and too small at the same time. I think Mini ITX would've been a better idea


  10. Another look at the 2 SATA ports provided by the JMS561.


    The same IC is inside Hardkernel's Cloudshell 2 thingy where users face serious SMART related issues. So let's do a quick check with an SSD connected to each SATA port:

    root@rock960:~# for i in a b ; do smartctl -d sat -x /dev/sd${i}; smartctl -l devstat /dev/sd${i} -d sat,12; echo -e "\n\n\n\n\n" ; done | curl -F 'f:1=<-' ix.io
    root@rock960:~# dmesg | tail -n5
    [   10.090856] rk_gmac-dwmac fe300000.ethernet eth0: Link is Up - 1Gbps/Full - flow control rx/tx
    [   89.067105] xhci-hcd xhci-hcd.7.auto: ERROR Transfer event for disabled endpoint or incorrect stream ring
    [   89.067959] xhci-hcd xhci-hcd.7.auto: @000000007bc32a30 00000000 00000000 1b000000 03078001
    [   89.162393] xhci-hcd xhci-hcd.7.auto: ERROR Transfer event for disabled endpoint or incorrect stream ring
    [   89.163274] xhci-hcd xhci-hcd.7.auto: @000000007bc32ac0 00000000 00000000 1b000000 03078001

    As http://ix.io/1o0p clearly shows the issue is still there. For the disk connected to the 'SATA1' port only bogus SMART data is returned. Shutting down the board, exchanging SATA cables so that now the EVO840 is connected to 'SATA1' and again: SMART readouts broken for the disk connected to this port: http://ix.io/1o0t


    I then wanted to create a btrfs mirror to test the same with drives busy but to my surprise Vamrs' kernel has no btrfs support enabled. Then tried an mdraid-1 (which I personally consider the most stupid disk setup ever but hey, it's only about a load test) but also no mdraid support in this kernel :(


    Ok, then continuing with individual ext4 filesystems on each SSD, running iozone -e -I -a -s 20000M -r 16384k -i 0 -i 1 -i 2 on each SSD and then again doing the smartctl queries.  No USB bus resets, performance also not affected but of course still bogus SMART readouts for one SSD and the above xhci error messages in the log.


    So both SATA ports still are fine from a performance point of view when combined with HDDs (since all HDDs are that slow that they are the bottleneck and not the JMS561 provided SATA port) but unfortunately there are functional limitations wrt the disk connected to the 'SATA1' port (no SMART health queries, you can start SMART selftests but since SMART status can not be queried no way to get selftest results without exchanging disks/cables).


    That being said Ficus with just one HDD connected to the 'SATA2' port is a great combination. I hope Vamrs looks into this, gets in touch with JMicron and checks whether this can be resolved with a firmware update.

  11. 4 hours ago, sfx2000 said:

    Once tests are done, looks like the cores are left on their own and cooking....


    'Cooking'? The 'performance' cpufreq governor is chosen, that's all. On an otherwise idle system there's not much consumption and temperature difference between idling at lowest or highest cpufreq OPP anyway.

  12. 2 hours ago, Superkoning said:

    you have a nice parquet floor!


    That's how craftsmen did it 120 years ago! :) All I did was removing ugly carpets decades ago ;)


    2 hours ago, Superkoning said:

    it looks like a nice board


    Let's wait until pricing is confirmed.


    This is definitely a server board which rises a couple of questions:

    • Enclosure? Especially if someone wants to use the PCIe slot (looking at this from above makes absolutely no sense at all)
    • Heat Dissipation? The huge PCB acts as a passive heatsink but what about demanding workloads? We all know that a fan alone is a joke compared to a good heatsink (then combined with just a little airflow)
    • What about the MCU stuff and the M.2 and mPCIe slots at the bottom?
  13. 9 minutes ago, NicoD said:

    Is there a command to display usb voltage in Linux?

    No. You need circuitry to measure this in the first place (only some old and boring Allwinner A20 devices provide this and the infamous Raspberry Pi has some circuitry to detect when input voltage drops below 4.63V +/- 10%)), then a driver interpreting the ADC data appropriately. If those two prerequisits are met the tool of choice to display this as a voltage value is close to irrelevant. But that's not the problem. The hardware is simply missing on majority of boards.

  14. 20 hours ago, tkaiser said:

    I think it's time to stop now, create a ficus.csc config and continue the tests with Armbian...


    Nope, lazy mode won (thanks to parted and resize2fs now using the eMMC's full capacity, then did do-release-upgrade to upgrade Vamrs' Xenial image to 18.04 LTS, then quickly checked partition 5 whether latest DT changes are present -- they are).  Then I took two SATA SSDs and one NVMe SSD (in Pine's PCIe to M.2 adapter), threw everything in a corner, connected the stuff and fired it up again:




    The huge fan is there to generate sbc-bench numbers now almost without throttling. Numbers here: http://ix.io/1nVS -- performance (not so) surprisingly identical to all other RK3399 thingies around but DDR3 memory results in slightly lower memory performance (negligible with almost every use case) and of course those RK3399 boards where we allowed 2000/1500 MHz so far show ~5% better CPU performance. The better cpuminer scores compared to RockPro64 with same 4.4. kernel here are just due to testing with Bionic on Ficus (GCC 7.3) vs. Debian Stretch there (GCC 6.3 -- with some tasks simply updating GCC you end up with whopping performance improvements for free)


    Not able to test the PCIe SSD since 'rockchip-pcie f8000000.pcie: PCIe link training gen1 timeout!' (full dmesg output). I guess there's still some work needed to get DT bits correctly for PCIe? Ok, let's forget about PCIe now (there won't be anything new anyway since PCIe lives inside RK3399 so as long as trace routing on the PCB is ok this RK3399 will perform 100% identical to every other RK3399 thingy out there)


    Now let's check USB situation. I'm a bit excited to have a look at the USB3 SATA implementation here since on the PCB itself there's a JMS561 controller (USB3-to-SATA bridge with support for 2 SATA ports, providing some silly RAID functionality but also PM mode. PM --> port multiplier). The two SSDs when connected with the JMS561 in PM mode appear as one device on the USB bus:

    root@rock960:~# lsusb
    Bus 004 Device 003: ID 1058:0a10 Western Digital Technologies, Inc. 
    Bus 004 Device 002: ID 05e3:0616 Genesys Logic, Inc. hub
    Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
    Bus 003 Device 002: ID 05e3:0610 Genesys Logic, Inc. 4-port hub
    Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
    Bus 002 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
    Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
    Bus 006 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
    Bus 005 Device 003: ID 0403:6010 Future Technology Devices International, Ltd FT2232C Dual USB-UART/FIFO IC
    Bus 005 Device 002: ID 1a40:0201 Terminus Technology Inc. FE 2.1 7-port Hub
    Bus 005 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
    root@rock960:~# lsusb -t
    /:  Bus 06.Port 1: Dev 1, Class=root_hub, Driver=ohci-platform/1p, 12M
    /:  Bus 05.Port 1: Dev 1, Class=root_hub, Driver=ehci-platform/1p, 480M
        |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/7p, 480M
            |__ Port 2: Dev 3, If 0, Class=Vendor Specific Class, Driver=ftdi_sio, 480M
            |__ Port 2: Dev 3, If 1, Class=Vendor Specific Class, Driver=ftdi_sio, 480M
    /:  Bus 04.Port 1: Dev 1, Class=root_hub, Driver=xhci-hcd/1p, 5000M
        |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/4p, 5000M
            |__ Port 3: Dev 3, If 0, Class=Mass Storage, Driver=uas, 5000M
    /:  Bus 03.Port 1: Dev 1, Class=root_hub, Driver=xhci-hcd/1p, 480M
        |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/4p, 480M
    /:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=ohci-platform/1p, 12M
    /:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=ehci-platform/1p, 480M
    • The JMS561 appears as '1058:0a10 Western Digital Technologies, Inc' device which is an indication that the chip runs with WD branded firmware (I'm not that excited). This chipset can be found in a lot of dual drive external USB3 thingies from various drive manufacturers but I thought we would see it here running with original JMicron firmware
    • There's a Genesys Logic USB3 hub on the board connected to RK3399's USB3 host port (the hub appears twice, once with product ID 0616 -- that's SuperSpeed most people call USB3 -- and once with ID 0610 -- that's Hi-Speed or incorrectly called 'USB2')
    • The JMS561 is behind the internal hub :(
    • We also have a 7-port Terminus Technology USB2 hub on the PCB
    • One FTDI UART adapter is internally connected to this hub (no idea what's that for -- maybe access to the 'MCU' that can be added to the board?)
    • I then used a 4th SSD in a VIA VL716 enclosure (USB-C) connected to the USB-C port just to end up with the same symptoms I already ran into when testing with USB-C on NanoPC-T4 and RockPro64 a while ago. Tons of error messages in the logs (updated dmesg). In the kernel messages the question 'Maybe the USB cable is bad?' appears and I would believe the kernel is right (I have 2 USB-C cables, the good one from Apple somewhere where I don't find it, the el cheapo providing those messages. I need to buy another one)

    The good news: there is SuperSpeed (USB3) on the USB-C connector and it's not behind the internal USB3 hub (BTW: Schematics for the board are here: https://dl.vamrs.com/products/ficus/docs/hw/). The bad news: all USB receptacles are behind one of the two internal USB hubs.


    So let's focus now on the 2 SATA ports provided by the JMS561.


    1) Samsung EVO840 connected to one of the SATA ports. Testing methodology exactly identical as outlined here: https://forum.armbian.com/topic/8097-nanopi-m4-performance-and-consumption-review/?do=findComment&amp;comment=61783. Only some minor slowdowns compared to NanoPi M4 (there the JMS567 USB-to-SATA bridge also being behind an internal USB3 hub). In other words: When HDDs are attached this is just fine since then the HDD is the bottleneck but neither the JMS561 nor the USB3 connection:

                                                                  random    random
                  kB  reclen    write  rewrite    read    reread    read     write
              102400       4    23192    26619    20636    20727    20633    26177
              102400      16    79214    89381    78784    79102    78889    88974
              102400     512   292661   295840   271486   277983   277481   300712
              102400    1024   321196   322169   305092   312765   312185   329470
              102400   16384   356091   356179   350200   360167   359817   357583
             2048000   16384   371761   374096   361612   361590   361724   374401


    2) Another Samsung added to 2nd 'SATA' port (now 2 disks connected to the JMS561). Running two independent iozone tests with 2GB test size at the same time. As expected we now run into 'shared bandwidth' and bus contention issues since both SSD are connected to the same JMS561 that is bottlenecked by the upstream connection to the SoC (one single SuperSpeed line that has to go through the USB3 hub). If we would talk about HDD the below numbers are still excellent since HDD are still the bottleneck with such a '2 disk' setup.

                                                                  random    random
                  kB  reclen    write  rewrite    read    reread    read     write
    PM851    2048000   16384   131096   132075   161149   162122   161282   131273
    EVO840   2048000   16384   151916   157295   176928   183874   186000   178285

    3) Now adding another Samsung SSD in an JMS567 enclosure to one of the USB3 receptacles (also behind the hub). As expected 'shared bandwidth' and bus contention issues matter even more now with 3 parallel iozone benchmarks running on each SSD in parallel. But the newly added Samsung EVO750 in an external enclosure finished way earlier:

                                                                  random    random
                  kB  reclen    write  rewrite    read    reread    read     write
    PM851    2048000   16384   112897   130875   125697   171855   175661   134626
    EVO840   2048000   16384   113541   132846   127664   172124   174311   137836
    EVO750   2048000   16384   214842   164890   328764   326065   197740   154398

    TL;DR: The 2 available SATA ports provided by a JMS561 in PM fashion are totally sufficient when you want to attach one or two HDDs. In case you need highest random IO or sequential IO performance exceeding the USB3 limit (with this 'behind a hub' topology this means ~365 MB/s) then your only choice is the PCIe slot once it works (either by using a PCIe based SSD there or a SATA or even SAS adapter)


  • Create New...