jungle_roger Posted May 3, 2016 Posted May 3, 2016 Hi, I noticed today that I'm getting those ARISC Errors again. I'm running the 5.05 server version which I ran apt-get upgrade on. I got the same when I tested out the new 5.10 image aswell. (Upgrading to 5.10 3.4.112 or using the new image wont let me compile the 'make scripts' either) [ 1424.305736] [ARISC ERROR] :message process error [ 1424.305763] [ARISC ERROR] :message addr : f004b840 [ 1424.305778] [ARISC ERROR] :message state : 5 [ 1424.305792] [ARISC ERROR] :message attr : 2 [ 1424.305805] [ARISC ERROR] :message type : 30 [ 1424.305818] [ARISC ERROR] :message result : ff [ 1424.305831] [ARISC WARING] :callback not install [ 1424.305849] [cpu_freq] ERR:set cpu frequency to 648MHz failed! Is this something I can fix myself or does it need to be done when compiling the Kernel?? Thanks Jungle
rodolfo Posted May 3, 2016 Posted May 3, 2016 I've experienced ARISC errors on Armbian_5.10 server for OPI ONE. The culprit was a faulty script.bin setting. Fix for OPI ONE : Login as root cp /boot/bin/orangepione.bin /boot/bin/orangepione.bin.oldbin2fex /boot/bin/orangepione.bin /boot/bin/orangepione.fexnano /boot/bin/orangepione.fex------------------------[dvfs_table]pmuic_type = 1pmu_gpio0 = port:PL06<1><1><2><1>pmu_level0 = 11300pmu_level1 = 1100max_freq = 1200000000min_freq = 648000000LV_count = 2LV1_freq = 1200000000LV1_volt = 1300LV2_freq = 648000000LV2_volt = 1100------------------------fex2bin /boot/bin/orangepione.fex /boot/bin/orangepione.bin et voilà ! ( settings valid only for OPI ONE )
tkaiser Posted May 3, 2016 Posted May 3, 2016 Apologies. That was my mistake (copy&paste gone wrong). Zador was able to nail it down, fix might come soon and you're able to fix it yourself using bin2fex/fex2bin already (no kernel exchange/compile needed). Everything outlined here: http://forum.armbian.com/index.php/topic/724-quick-review-of-orange-pi-one/?view=getlastpost
rodolfo Posted May 3, 2016 Posted May 3, 2016 fix might come soon and you're able to fix it yourself see previous ( 0.23567 seconds earlier ) post
tkaiser Posted May 3, 2016 Posted May 3, 2016 see previous ( 0.23567 seconds earlier ) post Nope, see the real solution. My assumptions back then when I defined initial dvfs settings were wrong (maybe also caused by a leading zero as now again). So a way better idea is to adopt zador's dvfs table that will be the 'fix' later since this allows lesser clockspeeds when not needed, leads to a cooler board and also more performance (when used without heatsink or in a small enclosure). The 2 relevant changes are: 480 MHz min instead of 648 MHz switch between 1.1V and 1.3V at 816 MHz and not 648 MHz as before
rodolfo Posted May 3, 2016 Posted May 3, 2016 Nope, see the real solution. My assumptions back then when I defined initial dvfs settings were wrong (maybe also caused by a leading zero as now again). So a way better idea is to adopt zador's dvfs table that will be the 'fix' later since this allows lesser clockspeeds when not needed, leads to a cooler board and also more performance (when used without heatsink or in a small enclosure). The 2 relevant changes are: 480 MHz min instead of 648 MHz switch between 1.1V and 1.3V at 816 MHz and not 648 MHz as before I've actually tested your proposed solution on OPI ONE and still got ARISC errors. I then tested the fix I posted on OPI ONE and got NO ARISC errors. Have you actually tested your solution on OPI ONE ?
tkaiser Posted May 3, 2016 Posted May 3, 2016 I then tested the fix I posted on OPI ONE and got NO ARISC errors. The 'fix you posted' are simply the older (and wrong) settings I came up with some months ago when I added support for Orange Pi One. Nope, I have not tested myself since my Orange Pi One arrived today at zador's home. And I've absolutely no doubt that if he says the new settings he came up with are ok, that they are ok. Next update will fix this.
zador.blood.stained Posted May 3, 2016 Posted May 3, 2016 @rodolfo Please post here your [dvfs_table] section if you are still getting ARISC errors. I won't be doing any more tests today, but if there are issues with new DVFS settings, I'll try to fix them tomorrow.
rodolfo Posted May 3, 2016 Posted May 3, 2016 @zador The fix ( actually reverting to 5.05 settings ) solves the problem. Working [dvfs_table] for OPI ONE (only two voltages defined) see previous post. Thanks for your help.
tkaiser Posted May 3, 2016 Posted May 3, 2016 I won't be doing any more tests today, but if there are issues with new DVFS settings, I'll try to fix them tomorrow. There are no issues: https://github.com/igorpecovnik/lib/issues/284#issuecomment-216655399
rodolfo Posted May 3, 2016 Posted May 3, 2016 @zador @tkaiser I've tested different dvfs-settings ( 2 - 5 levels ) for OPI ONE ( HDMI graphics session, remote graphical x2go-session with youtube-video running in iceweasel, htop average load 2.5 ). The following settings produced the lowest temperatures under comparable loads /etc/default/cpufrequtils------------------------ENABLE=trueMIN_SPEED=480000MAX_SPEED=1200000GOVERNOR=interactive------------------------ /boot/bin/orangepione.fex------------------------[dvfs_table]pmuic_type = 1pmu_gpio0 = port:PL06<1><1><2><1>pmu_level0 = 11300pmu_level1 = 1100max_freq = 1200000000min_freq = 480000000LV_count = 3LV1_freq = 1200000000LV1_volt = 1300LV2_freq = 1008000000LV2_volt = 1100LV3_freq = 480000000LV3_volt = 1100------------------------ cpufreq-info -------------------- ..... analyzing CPU 3: driver: cpufreq-sunxi CPUs which run at the same hardware frequency: 0 1 2 3 CPUs which need to have their frequency coordinated by software: 0 1 2 3 maximum transition latency: 2.00 ms. hardware limits: 480 MHz - 1.20 GHz available cpufreq governors: conservative, ondemand, userspace, powersave, interactive, performance current policy: frequency should be within 480 MHz and 1.20 GHz. The governor "interactive" may decide which speed to use within this range. current CPU frequency is 480 MHz. cpufreq stats: 60.0 MHz:0.00%, 120 MHz:0.00%, 240 MHz:0.00%, 312 MHz:0.00%, 408 MHz:0.00%, 480 MHz:66.91%, 504 MHz:0.02%, 600 MHz:0.08%, 648 MHz:0.00%, 720 MHz:0.01%, 816 MHz:0.00%, 912 MHz:0.00%, 1.01 GHz:20.89%, 1.10 GHz:0.38%, 1.20 GHz:11.71%, 1.30 GHz:0.00%, 1.34 GHz:0.00%, 1.44 GHz:0.00%, 1.54 GHz:0.00% (2783)-------------------- There were no ARISC messages with these settings.
zador.blood.stained Posted May 4, 2016 Posted May 4, 2016 LV2_freq = 1008000000 LV2_volt = 1100 It may be possible to run with 1008MHz@1.1V, but according to linux-sunxi wiki it's not recommended, so default Armbian DVFS table will look like that.
tkaiser Posted May 4, 2016 Posted May 4, 2016 HDMI graphics session, remote graphical x2go-session with youtube-video running in iceweasel, htop average load 2.5 Unfortunately this pretty lightweight workload isn't relevant when trying to push hardware limits. I for example have one Orange Pi PC that runs with these settings stable even when torturing it: http://linux-sunxi.org/User:Tkaiser#Headless_without_connected_USB_peripherals_and_4.5V_DC-IN The very same settings lead to bit flips on a colleague's Orange Pi PC and are therefore not useable as general settings. So while your settings might work for you we have to take into account that other devices behave different and always leave some safety headroom. Regarding torturing: At least this procedure has to work flawlessly to be sure that your settings work for your device: http://linux-sunxi.org/Hardware_Reliability_Tests#CPU Bit flips are nasty especially when they only occur under special circumstances. So we have to take sane defaults with higher voltages at specific dvfs operating points than those we tested to work reliably on our own devices.
rodolfo Posted May 4, 2016 Posted May 4, 2016 @zotac The new default Armbian dfvs_table looks perfectly fine as I've already successfully tested these settings. @tkaiser Your use cases of "torturing poor little boards" of course merit safer default settings. So far I've not seen any problems in running typical target loads with the simpler settings for OPI ONE ( migth be different for OPI PC ). Thanks for the links, I'm tempted to start a little bonfire with a spare board. Thank you both for your excellent work in converting OrangePi bricks into something useful.
Set3 Posted May 6, 2016 Posted May 6, 2016 Ah that explains my blistering hot One after upgrade indeed, it's ok now with the 2 LV_count fix from Rodolfo, thanks ! So when will it be automatically rolled out when I want to upgrade another OPiO ? How does that work ?
rodolfo Posted May 7, 2016 Posted May 7, 2016 @Set3 Glad to hear your Opi One stopped burning. The "fix" is actually just adjusting minimum frequency and reverting to previous ( tkaiser ) settings. The next Armbian release will feature a generally optimized dvfs_table for OPI ONE. It should be fixed with your next upgrade. Search for "script.bin", "fex" on this or the the sunxi.org site for further info on board definitions.
tkaiser Posted May 7, 2016 Posted May 7, 2016 Ah that explains my blistering hot One after upgrade indeed, it's ok now with the 2 LV_count fix from Rodolfo, thanks ! C'mon guys. The whole issue is exactly none. Idle temperature increases by 10°C, temperatures under load aren't affected at all, a few error messages were thrown out. That's all. And people start to get crazy and spread bullshit like 'burning boards'. Really annoying! And you didn't fix anything you just reverted back to before the update and now use wrong settings since they base on (my) wrong assumptions made a few months ago. The new settings you could've adopted instead will show superiour behaviour. Stop whining now, use your boards as you did before, the next update will fix it. 1
Set3 Posted May 12, 2016 Posted May 12, 2016 Ok, clear. Sorry if you feel offended, that was not the intention.
rodolfo Posted May 13, 2016 Posted May 13, 2016 I've retested different settings for [dvfs_table] and actually noticed no different behaviour for 2 or 5 levels. There are only two voltages anyway, so the min, max and voltage change frequencies are all that matter. I've put some more load on the running system and tested the impact of the governor. By default. the governor is set to "interactive", a supposedly optimized "ondemand" governor adjusting frequency levels to loads. The "interactive" governor running on the OPI ONE behaved more like "performance", hardly ever switching from max.frequency and producing rather high cpu-temperatures ( no drama at all, I was just joking about the towering inferno boards , mea culpa, mea maxima culpa .....). A very noticeable change happened when switching to "ondemand" governor. Suddenly, the whole frequency range was being used, temperature dropped by 9°C and current draw was reduced by 50mA. With previously mentioned ( or soon to be released in Armbian ) fex-settings and a small heatsink on the OPI ONE the board behaves very nicely under the testing load ( x2go-server, flash dongle file-server with 2 parallel 4G scp network file-copies wireless and wired , modest browsing in iceweasel ) temperatures 39 (idle) , 59 (typical load) and 79 ( lots of stress ) when running governor "interactive" temperatures 39 (idle) , 50 (typical load) and 77 ( lots of stress ) when running governor "ondemand" To test governor "ondemand" : sudo echo ondemand>/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor To revert to governor "interactive" : sudo echo interactive>/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor To configure default governor "ondemand" : sudo nano /etc/default/cpufrequtils------------------------ENABLE=trueMIN_SPEED=480000 MAX_SPEED=1200000GOVERNOR=ondemand ------------------------ To check board temperature and current frequency : sudo watch cat /sys/class/thermal/thermal_zone0/temp sudo watch -n .1 cpufreq-info -w
tkaiser Posted May 14, 2016 Posted May 14, 2016 I've retested different settings for [dvfs_table] and actually noticed no different behaviour for 2 or 5 levels. There are only two voltages anyway, so the min, max and voltage change frequencies are all that matter. Sure, adding more dvfs operating points when only 2 voltages are available is a bit pointless. Maybe we revert back to just 2 (but still using 816 MHz to switch between voltages and not 648 MHz as before). Since you were able to use 1008 MHz with 1.1V and others also reported they get 1200 MHz stable with 1.1V I already thought about providing a script to automatically test this out. Main problem: users really love to fool themselves and reliably/really testing higher clockspeeds using cpuburn-a7 and cpufreq-ljt-stress-test (or Linpack -- see below) would require already a small fan since otherwise throttling might occur. Therefore such a script has to monitor /sys/devices/system/cpu/cpu0/cpufreq/stats/total_trans to be able to confirm whether throttling occured or not and to inform the testing user that his setup is inappropriate. When we optimised dvfs operating points for A64 back in March we found a specific Linpack benchmark as the perfect tool to detect undervoltage (since board does not already crash but the Linpack bench throws errors since it detects data corruptions) So we would need a kernel able to use also 1248, 1152, 1104, 1056, 960, 912 and 864 MHz as cpufreq steps (since we want to check reliability as fine graded as possible) and a script that installs Linpack patches the fex file so that one time only 1.1V and the other time only 1.3V will be used restarts the board automatically after adding itself to /etc/rc.local then starting reliability testing with 816 MHz clockspeed using Linpack and increasing clockspeed by 48 MHz incrementally until Linpack fails or /sys/devices/system/cpu/cpu0/cpufreq/stats/total_trans shows throttling then exchanging script.bin (now using 1.3V) using this specific clockspeed where 1.1V doesn't work reliably any more and testing through remaining cpufreq operating points until Linpack fails or throttling occurs If throttling occurs the user has to be informed that it's necessary to ensure appropriate heat dissipation (using heatsink and most probably an additional fan) since it's really necessary to test through the clockspeeds under full load. As a result you get the ideal cpufreq to switch between 1.1V and 1.3V the maximum cpufreq useable with just 1.3V (I would suspect most Orange Pi One/Lite and NanoPi M1 out there would be able to run reliably with 1248 or even 1296 MHz) And then it's up to the user. If he thinks data corruption and bit flips are nice then he can use these values otherwise subtracting 48 MHz from both dvfs operating points is the smarter move. But this way we could provide a mechanism every user could check his specific board for the real hardware limits (the VDD_CPUX voltage accuracy might differ between different boards and of course H3's ability to run reliably at different dvfs operating points). The same might be possible with the other H3 boards that use the fine graded programmable SY8106A voltage regulator to generate a board specific 100% optimal dvfs table. And maybe switching between 1.1V and 1.3V on the One is possible by writing to registers without exchanging script.bin and a reboot. Regarding your observations regarding cpufreq governor: 2 years ago I also preferred ondemand (on A20 boards) but only with the necessary settings regarding IO activity. In the meantime I came to the conclusion that interactive works better. But maybe different BSP kernels behave differently here (sun7i vs. sun8i for example). But IMO it's impossible to draw any conclusions without setting up real monitoring before. Easy with Armbian since it takes only 5-8 minutes to get RPi-Monitor installed.
rodolfo Posted May 14, 2016 Posted May 14, 2016 @tkaiser Thanks for detailling the rationale for H3 optimizations. IMHO the OPI ONE/LITE merit simple foolproof settings appropriate to running the boards within safe limits. A well-performing cheap little powerhouse without overclocking, overvolting, throttling, fans... running on stable Armbian. OPI ONE / (and probably LITE from the specs) run well with 2 dvfs operating points. Switching between 1.1V and 1,3V at 816 MHz ( lower than tested 1008 MHz ) should embrace any board quality. As I've mentioned before, using governor "ondemand" instead of "interactive" seems appropriate for OPI ONE when testing with typical and even more when testing with higher loads. Boards with a more sophisticated voltage/frequency control definitely merit appropriate testing/monitoring/optimization tools. For OPI ONE/LITE it is probably an overkill. By keeping it simple and using the simple boards well within their limits, the true power of these tiny jewels is unleashed : pleasing performance at a ridiculous price.
tkaiser Posted May 15, 2016 Posted May 15, 2016 For OPI ONE/LITE it is probably an overkill. Quite the opposite. These boards (NanoPi M1 as well) would benefit the most from such a reliability testing allowing them to both exceed 1200 MHz as maximum cpufreq and enter 1.1V state earlier. On passively cooled H3s this might lead to a performance increase by probably 25% under load. The only problem with this 'auto tuning' approach might be the user in question not willing to understand that he temporarely has to increase heat dissipation to test through all available dvfs operating points even if he does not want to use a fan in normal operation mode (for example have a look what I had to do to test out higher cpufreqs with A64 -- a simple fan should suffice to test H3 with heatsink and up to 1296 MHz). Adding few more cpufreq operating points shouldn't hurt at all so I opened a Github issue to discuss adding more with the other devs (based on experiences with Pine64 this might already increase full load performance by ~5% absolutely for free since throttling starts to work better). BTW: I also disagree regarding your cpu governor findings
rodolfo Posted May 15, 2016 Posted May 15, 2016 Adding few more cpufreq operating points shouldn't hurt at all so I opened a Github issue to discuss adding more with the other devs (based on experiences with Pine64 this might already increase full load performance by ~5% absolutely for free since throttling starts to work better). BTW: I also disagree regarding your cpu governor findings Adding few more cpufreq operating points sounds great. Governor "ondemand" used more frequencies than "interactive" when tested, leading to cooler running and less current draw.
tkaiser Posted May 15, 2016 Posted May 15, 2016 Governor "ondemand" used more frequencies than "interactive" when tested, leading to cooler running and less current draw. Unfortunately this is one of the many areas where tests and real world situations might differ. The problem with ondemand is that it takes some time to adjust the clockspeeds in both directions so while the cpufreq driver is still busy deciding whether to increase clockspeed or not with interactive you would've been already at maximum cpufreq (and the specific task maybe already done). Same applies to clocking down. And again: When using ondemand it's important to set io_is_busy to 1 otherwise IO intensive tasks will be way slower and it's also very important to set minimum cpufreq to a rather high value otherwise the system will behave always sluggish since crawling from lowest cpufreq to the highest takes too much time. Please see also the 'race to idle' concept mentioned here: http://linux-sunxi.org/Cpufreq#The_.22performance.22_governor
help40 Posted May 16, 2016 Posted May 16, 2016 I do not know why , but after applying the update YESTERDAY temperature increase in several graduses, nothing other changed ... (only aptitude upgrade... ) strange ? or ?
tkaiser Posted May 16, 2016 Posted May 16, 2016 I do not know why , but after applying the update YESTERDAY temperature increase in several graduses, nothing other changed ... (only aptitude upgrade... ) strange ? or ? No, quite normal since I introduced a bug with 5.10 -- I moved your post from the 'Security Alert' thread into this thread so you can follow the manual fix instructions as outlined here and get also a lot of basic knowledge for free above. Issue will be fixed with next Armbian release.
rampage Posted May 22, 2016 Posted May 22, 2016 Ok, I have decided to start my orange pi one story with armbian.. and the latest version of it, is not working due to this cpu freq. looping issues. Luckily, I was able to find previous version (5.05) and its running ideally with this board (thanks to developers). So, waiting for an official fix included to the latest distro.
Recommended Posts