TijuanaKez

Linux 6.1.63-current-rockchip64

Just chiming in here to say that after updating Armbian via OMV6 backend some time after my last post in Sep last year, my unit has remained stable for over 3 months. No idea why. Perhaps a magic fairy helped us. Happy days!

Yeah, totally hosed my whole setup. Can't ssh in anymore, and the serial output seems to have stopped working so it's a pretty pink paperweight till further notice

Thanks for adding that info. As for updates, I was actually thinking I was in the clear. The unit had stayed up without a hitch since my last post over a month ago and seemed impervious to putting it through it's paces. That was with 5.15.93-rockchip64 and 400-14000 ondemand governor. Sorry to say though I ran the auto update in the OMV6 backend that contained some Helios64 specific updates. At the time I thought great, someone, somewhere has made some tweaks! But now my unit is completely hosed. I can't even boot it to a state where I can ssh in and see what's up. TBH, I don't really understand where these updates came from. Are these coming from the community? Or are the being autogenerated by the Armbian build process, perhaps with old and outdated code that contains issues because the Helios code is not maintained? If so, anyone got tips on rolling back the kernel / updates? TBH I'm not even sure if the Helios64 specific updates were to do with the kernel or something else.

Interesting that yours doesn't freeze up on Segmentation fault etc. I also have a suspicion that the amount and kind of drives installed as this has an effect on power drawer and possibly integrity of the 12V and 5V rails. Grounds for the suspicion: Suggestion to raise VDD to 0.95V to improve stability as done with other Rockhip RK3399 boards Preventing the higher CPU frequencies makes it more stable. High CPU freqs are more demanding on power. The fact they shipped the unit with big electrolytic capacitors soldered to the end of wires on the Sata loom. Seems like a way to help smooth power rails without having to redesign the board. Various other hints here on the subject of VRMs, cpu governor settings, ramp time, latency etc in relation to the RK3399. https://forum.odroid.com/viewtopic.php?t=30303 https://github.com/u-boot/u-boot/commit/f210cbc1f3d4f84baa595d83933369b4d6a529ea https://github.com/u-boot/u-boot/commit/5a6d960b222203e60ab57e19b3eb7b41c24b564b https://wiki.t-firefly.com/en/Firefly-RK3399/driver_pwm.html http://patchwork.ozlabs.org/project/uboot/patch/20191128061433.1952869-2-anarsoul@gmail.com/ So, @mrjpaxton interested to know how many drives you have installed, and what type (5400, 7200, or SSD)? Also a possibility is running fewer tasks on the unit means the CPU has less demand and the governor may by rarely, if at all, trying to switch into the higher problematic frequencies. update: my unit still hasn't frozen again since last post with the 4000-14000 omdemand so things are looking pretty stable. so far so good anyway.

@mrjpaxtonI defintely bulit the unit corretly. As I said in the post, I had a %100 stable unit on OMV5 with the image listed in the wiki for years. Paying attention to this forum though, it's clear stability issues have been long running for many users and from what I can gather, the root problem is to do with cpu and power management. Quite a complex subject, something that PC overclockers would know more about than me. @snakekick thanks for chiming in too. Good info. Collecting info as I find it, it does seem like preventing the CPU from hitting the high end is the best way to go (contrary to other posts here). Still running on-demand 400-14000 now on @bunducafe's info, and it's more or less stable. As it's only happened once so far since changing to that setting, I can't even confirm it wasn't some other insult to the system

@mrjpaxton Thanks for the info. If I get some time I may try that out. Really though I'd be happy to freeze the kernel now if it would remain stable. The main purpose of the unit for me is running Nextcloud so whatever it takes to keep that running and the unit staying up. Btw, is your unit currently stable? If so can you share your CPU governor settings and any other tweaks you may have made? Thanks!

Mine never reboots, it only 'freezes' sporadically on the default cpu freq/governor settings and requires me to physically press the reset button. TBH a reboot would be better as I dont need to be in the house to fix it. Did you try disabling the suspend modes I mentioned a few posts back?

@mrjpaxton Thanks for chiming in. The TerreMaster does indeed have alot in common with the Helios64, and it does have the bonus of 10gbe. At over A$1000 (probably AUD$1100 with postage) it's it's more than double what I paid for the Helios full bundle, and the Helios was already a bit of a stretch philosophically for what a NAS adds to my life. Add to that the caveats you mentioned...... Likewise the 'build a small PC' type NAS always seemed I'd be entering into the power consumption equivalent of just leaving my main PC on 24/7 (which has plenty of sata and 10gbe already). When the PI3 then 4 came around, it seemed like the perfect thing for a NAS. The right amount of CPU/ram, the small form factor, low power consumption, low price, easy to get OMV on it etc. Just sadly no way to get real sata on it. With a Pi4 4GB costing me about A$100, it seemed adding *with* real sata for a modest price increase shouldn't be too much to ask for. Why doesn't someone make it I thought? Well they did, it was the Helios board. And it was awesome. And even though I dont care much about the case for something that'l probably be out of sight (I can 3D print), the hot pink got me Fast forward to now, when such things should be getting cheaper, TerraMaster think A$900 is appropriate to add to a raspberry pi 4 equivalent SBC to add 5 sata ports, 10gbe, and a plastic case. Not feeling any purchase urges at all. @bunducafe On a positive note, I've been running with your governor settings (ondemand - 4000 - 14000) and it does seem pretty stable. I've had to reset it once in 10 days, but I think I can live with that.

Hmm Interesting. I've been running some tests with different CPU governor settings but still haven't found stability with the ondemand governor (or shedultil for that matter) Out of curiosity, what made you limit the on demand governor to 1400? Most of the notes on this issue seem to say it's not the frequency, but the constant switching of frequencies that's the problem. I'll try with your settings and see how long mine stays up. I should note that I did take one 7200 drive out, but have since put a small SSD in it's place. Another thought that sprung to mind to keep the stability of performance governor, but not waste too much power would be to add some echo > scaling_max_freq commands as cron jobs to clock the CPU right down during part of the day when the Helios is usually not doing much other than staying alive.

These sleep/susepnd modes are seperate from the hard drive power settings. You can still set power/spin down in OMV. I't the Suspend to RAM state that is mentioned. Which probably means most of the states wont work by extension. CURRENT branch (Linux Kernel 5.10) Shutdown: OK Reboot: OK Suspend to RAM: Not Supported. USB host controller refuses to enter suspend mode Wake On LAN: Not Supported. Suspend still have issue

Of note also are the big electrolytic capacitors soldered onto long wires on the Sata power connector harness. Again just speculation, but this could have been an attempt to add extra power filtering to the boards after they'd been printed if a potential problem was identified later. I may experiment with upgrading these and check power stability on the rails when the drives are in use with a scope. For your random *reboot* issue. I would also look at suspend states. As of last updates, the suspend issues were still never resolved (see here), in which case unless I'm missing something, suspend should be disabled. But I think it's enabled by default for this board in Armbian 22 rockchip64. I think when it tries to enter one or more of those states, it fails and reboots. You can check with sudo systemctl status sleep.target suspend.target hibernate.target hybrid-sleep.target To Disable sudo systemctl mask sleep.target suspend.target hibernate.target hybrid-sleep.target After disabling suspend states, my unit never reset on its own again. But then I inherited the KP freeze instead. It's possible only particular states need to be disabled and e.g sleep can stay enabled, I'll keep testing. Fingers crossed, with fewer power hungry drives, and offending suspend disabled, the unit should stay up indefinitely.

Thanks for that info! 15-20 days I could live with, especially if by reboot you mean it restarts and doesn't freeze and need to be manually reset. Update from my own testing: I recently pulled one drive out so it's back to 4 bays out of 5. Since then it's been rock solid for over 2 weeks which is vastly different to before pulling the drive. I currently have CPU locked to 14000 in performance profile (but that didn't seem to help before pulling out this drive). If it continues to stay solid I will test again set back to on-demand full range. Being that some users report solid operation, and others have seen instability from the start (even on the old kernel), it would be interesting to see how many drives everyone has plugged in, and what type. I can see from quickly scanning back over some of the thread here that many users complaining about instability seem to be using all 5 bays with 3.5inch drives. Add to this the VDD boost for RK3399 mentioned here, the instability is feeling like it's current draw / PWM related. Maybe people know this already and I missed it. Just speculating, and I'm definitely our of my range of knowledge here, but perhaps 5x 7200RPM drives drawing current simultaneously and on-demand governor switching up to 18000 its tripping it somehow. Other *possibly* related reading material. https://forum.odroid.com/viewtopic.php?t=30303 https://github.com/u-boot/u-boot/commit/f210cbc1f3d4f84baa595d83933369b4d6a529ea https://github.com/u-boot/u-boot/commit/5a6d960b222203e60ab57e19b3eb7b41c24b564b https://wiki.t-firefly.com/en/Firefly-RK3399/driver_pwm.html http://patchwork.ozlabs.org/project/uboot/patch/20191128061433.1952869-2-anarsoul@gmail.com/

whoops! Sorry for that mistake. This is Armbian forums after all, I just dont really understand the inner workings of how the Armband is related to Debian. Just going off this from Kobol website. Yes, well, well aware of how things went with Kobol and that this hardware is no longer supported officially. This being the unmaintained section though (and because I know of no where else to ask) I was hoping to reach out any remaining Helios64 *users*. Its never my intention to bother Armbian developers to help fix unmaintained hardware. Hope you understand where I'm coming from. I'm not ready to give up on this board yet as I still see nothing that compares features wise (number of sata ports), power consumption wise, size, form factor, and well...I own one. Surely there are other users still using theirs?

If so how'd ya do it!?!? My personal experience had been a rock solid device on the original official Debian Armbian image (Linux Kernel: 5.10), and OMV5. It never went down unless I manually rebooted, and I never needed any CPU governor changes, voltage changes or otherwise to keep it stable. Reading so much stuff about OMV5 being end of life made me pull the trigger on upgrade to OMV6 along with the kernel upgrade that OMV did automatically, at the time unaware of the state of Kobol and Armbian support. Now I get about 1-2 days on average before it KPs and I have to hit the reset button. Tried the CPU underclocking, performance setting, VDD boost and various combinations but to no avail. So just reaching out to any other users here, is anyone still rocking a stable unit in 2023? Did you have to revert to Buster / OMV5? Are there any other tweaks to try that I've missed other than CPU clock/profile/vdd tweak? Is there any way to get any information on why it might be happening? Or is the only way to have a hardware serial console open 24/7 via a usb cable?