gprovost

  • Content Count

    497
  • Joined

  • Last visited

Reputation Activity

  1. Like
    gprovost reacted to ShadowDance in SATA issue, drive resets: ataX.00: failed command: READ FPDMA QUEUED   
    Hey, sorry I haven't updated this thread until now.
     
    The Kobol team sent me, as promised, a new harness and a power-only harness so that I could do some testing:
    Cutting off capacitors from the my original harness did not make a difference The new (normal) harness had the exact same issue as the original one With the power-only harness and my own SATA cables, I was unable to reproduce the issue (even at 6 Gbps) Final test was to go to town on my original harness and cut the connector in two, this allowed me to use my own SATA cable with the original harness and there was, again, no issue (at 6 Gbps) Judging from my initial results, it would seem that there is an issue with the SATA cables in the stock harness. But I should try to do this for a longer period of time -- problem was I didn't have SATA cables for all disks, once I do I'll try to do a week long stress test. I reported my result to the Kobol team but haven't heard back yet.
     
    Even with the 3.0 Gbps limit, I still occasionally run into this issue with the original harness, has happened 2 times since I did the experiment.
     
    If someone else is willing to repeat this experiment with a good set of SATA cables, please do contact Kobol to see if they'd be willing to ship out another set of test harnesses, or perhaps they have other plans.
     
    Here's some pics of my test setup, including the mutilated connector:
     

  2. Like
    gprovost reacted to qtmax in Helios4 performance concerns   
    I found the offending commit with bisect: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=5ff9f19231a0e670b3d79c563f1b0b185abeca91
     
    Reverting it on 5.10.10 restores the write performance, while not hurting read performance (i.e. keeps it high).
     
    I'm going to submit the revert to upstream very soon — it would be great if it also could be applied as a patch to armbian.
     
    (I would post a link here after I submit the patch, but this forum won't allow me to post more than one message a day — is there any way to lift this limitation?)
     
    @gprovost, could you reproduce the bug on your NAS?
  3. Like
    gprovost got a reaction from 0x349850341010010101010100 in Feature / Changes requests for future Helios64 board or enclosure revisions   
    RK3399 has a single PCIe 2.1 x 4 lanes port, you can't split the lanes for multi interfaces unless you use a PCIe switch which is an additional component that would increase a lot the board cost.
     
    That's clearly the intention, we don't want either to restart form scratch the software and documentation :P
     
    Yes this will be fixed, it was a silly design mistake. We will also make the back panel bracket holder with rounder edges to avoid user to scratch their hands :/
     
    We will post soon how to manage fan speed based on HDD temperature (using hddtemp tool), that would make more sense than current approach.
    You can already find an old example : https://unix.stackexchange.com/questions/499409/adjust-fan-speed-via-fancontrol-according-to-hard-disk-temperature-hddtemp
     
  4. Like
    gprovost reacted to Lore in Hanging on kernel load after Shutdown/Reboot/Kernel panic   
    I have run into this issue multiple times and the process flow is this:
     
    Flash image "Armbian_20.11.6_Helios64_buster_current_5.9.14.img" to a micro SD card.
    Run through the config (installing OMV with SAMBA)
    Kick over to the OMV to configure the share / media so its available on my network
    Runs for some time (this last install was running for maybe a few weeks before it triggered a Kernel panic)
    Power off.
    Power on.
    Board boots, hangs on "Starting Kernel..."
     
    Is there some trick I am missing to get this thing to actually boot / reboot correctly?
     
    Troubleshooting info:
    No jumpers are shorted.
    I had a similar issue with a previous img as well.
  5. Like
    gprovost reacted to qtmax in Helios4 performance concerns   
    I have a Helios4 (2nd batch) with 4 HDDs, and before I started using it, I decided to test its performance on the planned workload. However, I got quite not satisfied with the numbers.
     
    First, the SSH speed is about 30 MB/s, while the CPU core where sshd runs is at 100% (so I suppose it's the bottleneck).
     
    I create a sparse file and copy it over SSH: helios4# dd if=/dev/null bs=1M seek=2000 of=/tmp/blank With scp I get up to 31.2 MB/s: pc# scp root@helios4:/tmp/blank /tmp With rsync I get pretty much the same numbers: pc# rm /tmp/blank pc# rsync -e ssh --progress root@helios4:/tmp/blank /tmp/ blank 2,097,152,000 100% 31.06MB/s 0:01:04 (xfr#1, to-chk=0/1) (I tested both, because with another server, rsync over ssh runs faster - up to 80 MB/s, while scp is still at about 30 MB/s.)  
    Am I'm planning to use sftp/ssh to access the files, such speed is absolutely unsatisfactory. Is it possible to tune it somehow to be much closer to the line rate of 1 Gbit/s?
     
    The second question is about the best RAID level for this setup. My HDD's read speed is about 250 MB/s, and if I combine 4 of them into a RAID0, the read speed reaches 950 MB/s on another computer, but when I put these disks into the Helios4, the max read speed is about 350 MB/s. Given that the network speed is 125 MB/s, it doesn't seem useful to use any forms of striping, such as RAID0, RAID10 (far), etc., does it? What is the recommended RAID configuration for this system then, given that I have 4 HDDs?
     
    Would RAID10 (near) be better than RAID10 (far)? RAID10 (far) should provide the same read speed as RAID0 for sequential read, however, as on Helios4 the speed is far from 4x (it's just 1.4x), and the network line rate is anyway lower, maybe it makes more sense to optimize for parallel reads. RAID10 (near) will occupy only 2 disks on a single-streamed read (as opposed to 4 with RAID10 (far)), so a second reading stream can use two other disks and improve performance by avoiding seeking to different locations on a single disk. Does this layout make sense, or am I missing something?
     
    I'm also considering RAID6 that has better redundancy than RAID10 at the cost of lower speed and higher CPU usage. While lower speed might not be a problem if it's still higher than the line rate of 1 Gbit/s, CPU may be the bottleneck. As I see from the SSH test above, even such a simple test occupies 100% of one core and some part of the other core, so running RAID6 may increase the CPU demands and cause performance decrease.
     
    One more option is RAID1 over LVM that would combine pairs of disks, but I don't see any apparent advantage over RAID10 (near).
     
    Does anyone have experience running RAID10 (near/far), RAID6 or maybe other configurations with Helios4? What would you advise to build out of 4 HDDs 8 TB each if my goal is to have redundancy and read/write speeds close to the line rate?
  6. Like
    gprovost got a reaction from clostro in Feature / Changes requests for future Helios64 board or enclosure revisions   
    RK3399 has a single PCIe 2.1 x 4 lanes port, you can't split the lanes for multi interfaces unless you use a PCIe switch which is an additional component that would increase a lot the board cost.
     
    That's clearly the intention, we don't want either to restart form scratch the software and documentation :P
     
    Yes this will be fixed, it was a silly design mistake. We will also make the back panel bracket holder with rounder edges to avoid user to scratch their hands :/
     
    We will post soon how to manage fan speed based on HDD temperature (using hddtemp tool), that would make more sense than current approach.
    You can already find an old example : https://unix.stackexchange.com/questions/499409/adjust-fan-speed-via-fancontrol-according-to-hard-disk-temperature-hddtemp
     
  7. Like
    gprovost reacted to hartraft in Feature / Changes requests for future Helios64 board or enclosure revisions   
    Whilst i'm satisfied with my Helios64 as well, i would have liked:
    the original RK3399Pro as the NPU would have been useful in tensorflow for Photoprism (as a Google photos replacement). RK3588 would be even nicer but everyone is waiting on that chip more RAM, once Photoprism indexes photos, it easily runs out of memory. I limit the memory in docker now but its not ideal. Granted, there is an issue on their code so i just keep the container restarted each night NVMe for fast disk access Better HDD trays (already mentioned) the screw eyelets for the backpanel (1st pic below) not to be directly aligned with the eyelets for the backplane (2nd pic below) for easier assembly.
  8. Like
    gprovost reacted to Salamandar in Feature / Changes requests for future Helios64 board or enclosure revisions   
    I added voting options, please revote !
     
    EDIT: crap… We can't edit our votes, I did not expect that
  9. Like
    gprovost reacted to ebin-dev in Feature / Changes requests for future Helios64 board or enclosure revisions   
    The next thing I would buy is a drop in replacement board for the helios64 with:
    rk3588 ECC RAM  nvme PCI-e port 10GBase-T port
  10. Like
    gprovost reacted to clostro in Feature / Changes requests for future Helios64 board or enclosure revisions   
    - I don't see the point of baking a m.2 or any future nvme PCI-e port on the board honestly. Instead I would love to see a PCI-e slot or two exposed. That way end user can pick what expansion is needed. It can be an nvme adapter, can be 10gbe network, or SATA or USB or even a SAS controller for additional external enclosures. It will probably be more expensive for the end user but will open up a larger market for the device. We already have 2 lanes/10gbps worth of PCIe not utilized as is (correct me on this).
    - Absolutely more/user upgradeable and ECC ram support.
    - Better trays, you guys they are a pain right now lol. ( Loving the black and purple though )
    - Some sort of active/quiet cooling on the SoC separate from the disks, as any disk activity over the network or even a simple CPU bound task ramps up all the fans immediately.
    - Most important of all, please do not stray too far from the current architecture both hardware and firmware wise. Aside from everything else, we would really love to see a rock solid launch next time around. Changing a lot of things might cause most of your hard work to-date to mean nothing.
  11. Like
    gprovost reacted to yalman in Virtualization support   
    I found a hothotfix solution without knowing the (side) effects.
     
    # 0,1,2,3=Quad-core Cortex-A53 up to 1.4GHz
    # 4,5=Dual-core Cortex-A72 up to 1.8GHz
    root@helios64:~# cat /proc/cpuinfo |grep -e processor -e part
    processor    : 0
    CPU part    : 0xd03
    processor    : 1
    CPU part    : 0xd03
    processor    : 2
    CPU part    : 0xd03
    processor    : 3
    CPU part    : 0xd03
    processor    : 4
    CPU part    : 0xd08
    processor    : 5
    CPU part    : 0xd08
     
    # always assign the same cpu types to a VM. core 0,1 for the host system
    virsh edit VM1
    ...
    <vcpu placement='static' cpuset='2-3'>2</vcpu>
    ...
    virsh edit VM2
    ...
    <vcpu placement='static' cpuset='4-5'>2</vcpu>
    ...
  12. Like
    gprovost reacted to djurny in Helios64 - freeze whatever the kernel is.   
    Hi @SR-G,
    FYI I have been running Linux kobol0 5.9.13-rockchip64 #trunk.16 SMP PREEMPT Tue Dec 8 21:23:17 CET 2020 aarch64 GNU/Linux for some days now. Still up. Perhaps you can give this one a try?
    Distributor ID: Debian Description: Debian GNU/Linux 10 (buster) Release: 10 Codename: buster  
    /etc/default/cpufrequtils:
    ENABLE=true MIN_SPEED=408000 MAX_SPEED=1200000 GOVERNOR=powersave (The box runs idles with 'powersave' and will be set to 'performance' when performing tasks.)
     
    I do not know which non-dev kernel version this is? I assume it's 21.02 or 21.03?

    Groetjes,
     
  13. Like
    gprovost reacted to ShadowDance in Mystery red light on helios64   
    It's probably indicating a kernel panic. You could try limiting CPU speed and setting the governor as suggested in the linked topic. You could also try hooking up the serial port to another machine and monitor it for any output indicating why the crash happened.
     
  14. Like
    gprovost reacted to barnumbirr in Helios64 Ethernet Issue   
    No, don't need to perform fix if you're only ever going to use the 1Gbps interface AFAIK.
  15. Like
    gprovost reacted to Salamandar in Feature / Changes requests for future Helios64 board or enclosure revisions   
    Hi everyone !
    I'm really happy with my Helios64, running fine with no major issues.
    BUT
    This product can obviously be improved with future revisions, while keeping the same base. The Kobol guys told me to open a thread here to discuss what would be great improvement or new features.
     
    I'm starting with things off my mind, but please ask me to add things to this list. An explanation would be great in the comments.
  16. Like
    gprovost got a reaction from Salamandar in Ethernet 1Gbps often disconnects on Legacy kernel   
    Effectively there was an issue with /boot partition behind too small, it has been fixed in future build https://github.com/armbian/build/pull/2425
  17. Like
    gprovost reacted to Igor in ZFS dkms package upgraded to v2.0.1   
    Media info:
    https://arstechnica.com/gadgets/2020/12/openzfs-2-0-release-unifies-linux-bsd-and-adds-tons-of-new-features/
     
    Remarks: 32bit support is broken in Debian Stretch, no upgrade there. Also this upgrade is needed to bump kernel to 5.10.y
  18. Like
    gprovost got a reaction from clostro in No HDD1 when using M.2 SATA on SHARED port.   
    @Werner Hahaha yes we are working at improving the design ;-) More news soon.
  19. Like
    gprovost got a reaction from clostro in No HDD1 when using M.2 SATA on SHARED port.   
    @fr33s7yl0r Updated your title to make it more friendly.
     
    If we could have made it 6x SATA port at that time, with one port dedicated to the M.2 slot, you can be sure we would have done it.
    We could have taken a different approach using a PCIe Switch, but it would have increase significantly the board.
     
    Any board design is about finding the best recipe within a budget for a list of use cases with the less tradeoffs possible.
    The M.2 SATA port on our Helios64 was an addition to support more use cases to the product, even though yes it wasn't perfect that the port has to be shared.
     
  20. Like
    gprovost got a reaction from Werner in No HDD1 when using M.2 SATA on SHARED port.   
    @Werner Hahaha yes we are working at improving the design ;-) More news soon.
  21. Like
    gprovost reacted to Igor in ZFS on Helios64   
    ZFS was just upgraded to v2.0.1 https://armbian.atlassian.net/browse/AR-614 which is a preparation for pushing a kernel on and above 5.10.y. Upgrade was tested on Armbian Buster/Bullseye/Focal/Hirsute. My test ZPOOL was always loaded without any troubles.

    We only support upgrade troubles - for ZFS functional problems, please check / file a bug report here https://github.com/openzfs/zfs/issues
     
  22. Like
    gprovost reacted to fr33s7yl0r in No HDD1 when using M.2 SATA on SHARED port.   
    I literally registered to post this complaint here.
    How STUPID it is to have a M.2 SATA SHARED WITH HDD1!!
    IT DEFEATS THE WHOLE PURPOSE OF THIS ENCLOSURE. PERIOD!!!!
     
    My mistake was that I did not know what it meant that M.2 was SHARED before I purchased this enclosure and now, I have a need to expand beyond the measly 16GB of your eMMC and I found out THE PAINFUL WAY that IT IS NOT POSSIBLE because of the shared SATA port. IT"S BEYOND FRUSTRATION.
     
    YOU CANNOT EVEN IMAGINE HOW STUPID THIS DECISION OF YOURS IS.
    IT JUST LITERALLY DOWNGRADED YOUR NAS FROM AN EXCELLENT ENCLOSURE TO AN OVERPRICED JUNK THAT I HAD TO WAIT FOR FOR MORE THAN 6 MONTHS.
     
    AND THERE IS NO WAY TO RETURN IT TO YOU AND GET MY MONEY BACK.
     
    What should I do with my RAID now, once I want to install an SSD drive?
    Should I now use RAID drive as a slow storage and have symbolic links to make sure that I don't run out of space on eMMC?
     
    IF THERE WAS SOME WAY TO RATE YOUR ENCLOSURE SO NOBODY ELSE MAKES THE SAME MISTAKE I DID, I WOULD CERTAINLY DO THAT.
     
     
     
  23. Like
    gprovost reacted to Atticka in How to be sure that Helios64 or it's power supply not cause fire?   
    In general any electronic device should be plugged in to a proper electrical outlet with a circuit breaker.  The circuit breaker is critical in cutting the current in case of a short, therefore preventing a fire.
  24. Like
    gprovost reacted to aprayoga in Controlling SATA Power Rail   
    @allen--smithee
    if you need to modify the bootloader, you can start from these lines
    https://github.com/armbian/build/blob/master/patch/u-boot/u-boot-rockchip64-mainline/add-board-helios64.patch#L1671-L1687
    You can add parameter checking there. Maybe check certain U-Boot environment variable, maybe check for certain file in the filesystem.
    You could also remove it completely and modify /boot/boot.scr instead, to enable/disable the HDD power.
     
    Another thing to explore, you could remove the afromentioned u-boot function and also remove the power rail nodes from linux device tree
    https://github.com/armbian/build/blob/master/patch/kernel/rockchip64-current/add-board-helios64.patch#L239-L259
    and then when you need to enable it, just access the gpio from sysfs.
     
  25. Like
    gprovost reacted to devman in All SATA drives down after (coincidental?) update   
    @gprovostHit up the local computer mall for a replacement power supply.  Closest they had was 12v 10A to barrel, and then a barrel to 4-pin adapter.
     
    Anyways, everything checked out correctly, and no more errors.  Thanks!