Charlie Romer Posted May 9 Posted May 9 2 minutes ago, Igor said: We are on the same side. This is what I've been trying to say the whole time... I know many distros are small projects with no full time developers. I saw in this thread that it looked like you guys could use some help and given how new this board is I wasn't expecting anyone to just solve something for me. I wanted to jump in and help if and where I could. What did you expect? Truth or fake happiness and prompt resolving of your problem? The opposite literally. I figured that if I have the ability to help and hardware in hand that I should help (or at least offer to be a guinea pig) - I wasn't and haven't been looking for someone to solve this thing for me but literally offering my assistance as I happen to have the hardware and wanted to give back if I could. This whole conversation has felt like I've been gaslit by people who assume everyone posting for the first time on this forum must be demanding fixes for everything right now. That may be many (or even most) folks but it's not a great feeling when all you're doing is trying to offer help. 0 Quote
going Posted May 10 Posted May 10 @Charlie Romer Hi. Let's try to figure it out. 5 часов назад, Charlie Romer сказал: I figured that if I have the ability to help and hardware in hand that I should help Core panic - how is this problem solved? It is assumed that you have the kernel source code. You look at the kernel messages, and then using utilities like grep you find the function in the source code. The very function that could not cope and caused panic. This appears in the kernel panic messages. The first probable reason that led to this is that a patch was incorrectly applied to a file with this function. I.e. we are looking for all patches that modified this source code file. In fact, we will need to apply all the patches in the series to the source code in the core git repository. Apply as the "git am" command. We pay special attention to those patches that change our target file. Then we extract a series of our patches from the repository using a script and place it in the target patch folder. I.e., we replace the existing series with a new one, which is applied without offsets and fuzziness. The second reason is the changes in the upstream of the kernel that led to the situation our patch has aged. In this case, we are fixing this patch. We simply make our new changes to the fixation after applying the patch in the process of applying the series step by step. The complexity of the core panic situation in the Armbian project lies in the fact that it is impossible to reproduce this particular problem. Therefore, if you did everything as I wrote and made the necessary corrections from your point of view, then assembled the kernel and the panic repeated, you simply push your changes to your repository on github and only after that you can write here on the forum or ask a question here by specifying a link to your repository and the kernel version to which corrections have been applied. Only after that, the maintainer or the interested user will be able to check it. The third possible option is that the u-boot code does not match the kernel code due to their development. We may have to monitor this loader build process as well. @Igor Did I miss something? Where did the fixed kernel version go in these settings? How is the correct application of patches controlled today? 0 Quote
Igor Posted May 10 Posted May 10 14 minutes ago, going said: How is the correct application of patches controlled today? Hasn't changed. https://github.com/armbian/build/tree/main/patch/kernel Just symlinking of subfolder are perhaps adding some confusion. 0 Quote
going Posted May 10 Posted May 10 1 минуту назад, Igor сказал: Hasn't changed. https://github.com/armbian/build/tree/main/patch/kernel Just symlinking of subfolder are perhaps adding some confusion. I meant this line: KERNELBRANCH="tag:v6.1.69" which uniquely determines which version of the kernel patches are applied cleanly. I.e. to version 6.1.69 and not to the current 6.1.90 for today. 0 Quote
Igor Posted May 10 Posted May 10 1 minute ago, going said: which uniquely determines which version of the kernel patches are applied cleanly. I.e. to version 6.1.69 and not to the current 6.1.90 for today. Oh, yes. This must also work. or "commit:xxxxxxxxxxxxxxxxx" 0 Quote
rosbeef Posted May 30 Posted May 30 (edited) Hi ! i am in this situation: Orangepi 5plus 32gb ram , uboot on spi and root on nvme i experiment the crash on Armbian 24.5.1 Bookworm with Linux 5.10.160-legacy-rk35xx and 6.1.43-vendor-rk35xx and 6.8.10-edge-rk3588 too I would help you to resolve my bug but could you help me to send you all you could need. i recover some datas from the serial 409.837122] Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b [ 409.837821] CPU: 3 PID: 1 Comm: systemd Not tainted 6.8.11-edge-rockchip-rk3588 #1 [ 409.838494] Hardware name: Xunlong Orange Pi 5 Plus (DT) [ 409.838967] Call trace: [ 409.839191] dump_backtrace+0x94/0x114 [ 409.839538] show_stack+0x18/0x24 [ 409.839840] dump_stack_lvl+0x38/0xf0 [ 409.840173] dump_stack+0x18/0x24 [ 409.840473] panic+0x3b8/0x3f8 [ 409.840754] do_exit+0x898/0x9bc [ 409.841050] do_group_exit+0x34/0x90 [ 409.841376] copy_siginfo_to_user+0x0/0xc8 [ 409.841746] do_notify_resume+0x27c/0x1430 [ 409.842114] el0_da+0x8c/0x90 [ 409.842387] el0t_64_sync_handler+0xe4/0x158 [ 409.842772] el0t_64_sync+0x1a4/0x1a8 [ 409.843104] SMP: stopping secondary CPUs [ 409.843559] Kernel Offset: disabled [ 409.843871] CPU features: 0x0,c0000000,7002814a,2100720b [ 409.844345] Memory Limit: none [ 409.844629] ---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b ]--- other dump 279.842385] Internal error: Oops: 0000000096000004 [#1] PREEMPT SMP [ 279.842949] Modules linked in: macvlan nft_masq bridge snd_seq_dummy rfcomm snd_seq snd_seq_device nft_fib_inet nft_fib_ipv4 nft_fib_ipvb [ 279.843297] libarc4 gpu_sched drm_shmem_helper drm_exec rfkill_gpio rfkill dm_mod ip_tables x_tables autofs4 uas rockchipdrm drm_dma_hes [ 279.853805] CPU: 2 PID: 190 Comm: kworker/2:1 Not tainted 6.8.11-edge-rockchip-rk3588 #1 [ 279.854522] Hardware name: Xunlong Orange Pi 5 Plus (DT) [ 279.854995] Workqueue: events kernfs_notify_workfn [ 279.855439] pstate: a0400009 (NzCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--) [ 279.856057] pc : find_inode_fast+0x38/0xe4 [ 279.856430] lr : ilookup+0x8c/0x118 [ 279.856747] sp : ffff80008353bca0 [ 279.857044] x29: ffff80008353bca0 x28: 0000000000000000 x27: ffff800081f57980 [ 279.857686] x26: ffff800081d5e000 x25: ffff00010048ebb0 x24: ffff800081172078 [ 279.858327] x23: ffff0001008a5800 x22: ffff0007d9300c00 x21: ffff0001008a5800 [ 279.858968] x20: 00000000000027c8 x19: 90d48fefa3653def x18: ffff80008c523c58 [ 279.859609] x17: 0000000000000000 x16: 1fffe0002046dc41 x15: 00000000000002e9 [ 279.860249] x14: 0000000000000000 x13: 0000000000000030 x12: 0101010101010101 [ 279.860889] x11: 7f7f7f7f7f7f7f7f x10: fefefefefefefeff x9 : 7f7f7f7f7f7f7f7f [ 279.861530] x8 : 8080808080808080 x7 : ffff00010048ebb0 x6 : ffff000100aa7380 [ 279.862170] x5 : d83827dd7f7cc000 x4 : 0000000000000015 x3 : d9bf05c4657e16ae [ 279.862810] x2 : 00000000000027c8 x1 : ffff0007d9300c00 x0 : ffff0001008a5800 [ 279.863451] Call trace: [ 279.863675] find_inode_fast+0x38/0xe4 [ 279.864016] ilookup+0x8c/0x118 [ 279.864303] kernfs_notify_workfn+0x12c/0x220 [ 279.864698] process_one_work+0x160/0x3a8 [ 279.865064] worker_thread+0x32c/0x438 [ 279.865404] kthread+0x118/0x11c [ 279.865701] ret_from_fork+0x10/0x20 [ 279.866030] Code: aa0003f5 f9001bf7 d503201f 54000140 (f9402263) [ 279.866441] Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b [ 279.866448] SMP: stopping secondary CPUs [ 279.866573] ---[ end trace 0000000000000000 ]--- [ 279.866610] Kernel Offset: disabled [ 279.868331] CPU features: 0x0,c0000000,7002814a,2100720b [ 279.868799] Memory Limit: none [ 279.869076] ---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b ]--- Edited May 30 by rosbeef more spec 0 Quote
Pierre C. Posted May 30 Posted May 30 (edited) Hi all, Well, I've been having some issue with my OPi5+ and I managed to solve most of them so far. As to the issue at hand, I've been running with Armbian 24.2.1 Bookworm with Linux 5.10.160-legacy-rk35xx for a while on my 16GB no worries. As for the 32GB, I can't say for Armbian as I haven't tested. But with J-R's, my booting issues and memory issues went away when I replaced my PSU. It seems the 32GB needs a little more power than the 16GB, I replaced it with a proper 5v 6A PSU and all my worries went away, it boots and hasn't crashed yet ! It might work with an original 5v 5A, but mine was from Ali and might be over-rated as most is on this site ... Hope this might help someone... All my OPi5+ (16+32) are installed with a Force MP600 2tB Nvme Edited May 30 by Pierre C. 1 Quote
rosbeef Posted May 30 Posted May 30 (edited) Damn ! the opi5+ bundle from aliexpress includes a 5V-4A which should be short for a 32G+nvme. I will try a 5V-5A as i am not able to find a 5V-6A on amazon. Edited May 30 by rosbeef 0 Quote
Werner Posted May 30 Posted May 30 I use an mean well HRP-200-5...but for all my stuff https://media.it-tronics.de/Datasheets/Power_Supplies/MeanWell/HRP-200.pdf 1 Quote
rosbeef Posted June 2 Posted June 2 (edited) More current (5A instead of 4A) did not resolve the problem for me. I tried the vendor distro (debian) and problem is not solved. Writing from SD to emmc makes crash too. even if no usb connected. A simple copy command make debian crash. i will go to orangepi forum 😕 for help. Edited June 2 by rosbeef 0 Quote
Werner Posted June 2 Posted June 2 Voltage might be an issue. Most SBCs while rated for 5 volts prefer slight overvoltage, last but not least to compensate for voltage loss across wiring, connectors and the PCB itself. Even PSU sold by xunlong in combination with the OPi5, also while rated for 5v actually outputs about 5.2V. I verified this on an OPi5 month ago by measuring at the voltage source which was between 5.2 and 5.3 volts and the USB-A connector on the board which dropped to 5.0 volts when putting on a heavy CPU load. Since the PSU itself has way more than enough power voltage there were constant. Now think about what happens when your PSU only outputs barely enough wattage. Not only the voltage drop mentioned earlier will be there but also the voltage will break down at PSU level which increases the overall voltage drop to a (from a electronics perspective) crazy level. So if your input voltage is 5 volts only it will drop below this and malfunctioning is VERY likely then. I'd test this on the 5+ 32G model if I had one. But mine has 16G memory only. 0 Quote
usual user Posted June 2 Posted June 2 1 hour ago, Werner said: Voltage might be an issue. The issue is probably due to the fact that rk35xx SoCs have to carve out some memory areas above 16 GB as reserved. Access to those areas leads to crashes. Because no fully open source TF-A is available yet, current mainline U-Boot has mechanisms landed to take the area information from the closed source TPL. Chances are the firmware you're using hasn't backported this yet, because devices larger than 16GB are only now becoming widely available in the open market. 0 Quote
rosbeef Posted June 2 Posted June 2 1 hour ago, usual user said: Chances are the firmware you're using hasn't backported this yet, because devices larger than 16GB are only now becoming widely available in the open market. i don't need 32 for now, i jut bought it for the future. Should it be possible to lie on the size of the memory and only declare 16Gb usable ? 0 Quote
rosbeef Posted June 2 Posted June 2 3 hours ago, Werner said: So if your input voltage is 5 volts only it will drop below this and malfunctioning is VERY likely then. i don't have the tools to test the voltages and don't have a precise tunable powersource. as you seems to know about electronic can i ask you ? Is 6V will be too much overvoltage ? 0 Quote
usual user Posted June 2 Posted June 2 30 minutes ago, rosbeef said: Should it be possible to lie on the size of the memory and only declare 16Gb usable ? Setup mem=16G at kernel cmdline. 0 Quote
Werner Posted June 2 Posted June 2 1 hour ago, rosbeef said: Is 6V will be too much overvoltage ? yes, it is. 5.2V to .3 seems to be a reasonable value. 1 Quote
rosbeef Posted June 3 Posted June 3 (edited) Hum i saw that my power is 5.1v I removed the mvne and try to cp a big file, it crash too. Maybe the right way to explore is the bad memory managment. 20 hours ago, usual user said: Setup mem=16G I could not reduce visible memory, sorry i'm noob here. I put the mem=12G in the /boot/armbianEnv.txt file then "update-initramfs -u" I tried directly in /boot/boot.cmd on the bootagrs parameter too without success. , i read that too : Quote mem= is for manual override or prehistoric bootloaders not supporting atags or device tree. source https://github.com/linux-sunxi/u-boot-sunxi/issues/11#issuecomment-6358135 maybe there is an other way to shrink the memory or accept the 32G. Edited June 3 by rosbeef 0 Quote
Efe Çetin Posted June 3 Posted June 3 @rosbeef have you ever tried to update DDR and BL31 blobs? 5plus still uses old blobs that may cause the issue. You can update blobs by putting ``` DDR_BLOB='rk35/rk3588_ddr_lp4_2112MHz_lp5_2400MHz_v1.16.bin' BL31_BLOB='rk35/rk3588_bl31_v1.45.elf ``` to the https://github.com/armbian/build/blob/main/config/boards/orangepi5-plus.conf and compile the new uboot spi image/package or new image. 0 Quote
MichaIng Posted June 4 Posted June 4 (edited) Am 3.6.2024 um 10:13 schrieb rosbeef: I put the mem=12G in the /boot/armbianEnv.txt file then "update-initramfs -u" I tried directly in /boot/boot.cmd on the bootagrs parameter too without success. Either, you need to add it to the "extraargs=" (optionally "extraboardargs=") line in /boot/armbianEnv.txt, in case create those lines, if missing: extraargs=mem=16G Or, when adding it to /boot/boot.cmd directly, you need to generate /boot/boot.scr from it: mkimage -C none -A arm64 -T script -d /boot/boot.cmd /boot/boot.scr The initramfs does not need to be regenerated in any case. Edited June 4 by MichaIng 0 Quote
rosbeef Posted June 4 Posted June 4 3 hours ago, MichaIng said: extraargs=mem=12G line in /boot/armbianEnv.txt, ok thankyou, that makes debian boot with 12G effectively and big file (250GB tested) coping is fully working. So it discard a hardware power failure. Sorry @Werner Now i have a half working memory. And if i want to get my fully working 32G memory i have to try the @Efe Çetin solution. On 6/3/2024 at 1:11 PM, Efe Çetin said: DDR_BLOB='rk35/rk3588_ddr_lp4_2112MHz_lp5_2400MHz_v1.16.bin' BL31_BLOB='rk35/rk3588_bl31_v1.45.elf just let me time to compile the new armbian with those blobs Thanks all. 0 Quote
rosbeef Posted June 5 Posted June 5 @Eddie as @MichaIng explain me did you add the next in the armbienEnv.txt ? 23 hours ago, MichaIng said: extraargs=mem=16G 0 Quote
Eddie Posted June 5 Posted June 5 3 minutes ago, rosbeef said: explain me did you add the next in the armbienEnv.txt ? Yes. Added it to other arguments, like this: extraargs=fsck.mode=force fsck.repair=yes mem=16G free -h shows 16G after this. 0 Quote
MichaIng Posted June 5 Posted June 5 (edited) Since a board with 16G RAM practically has a little less than 16 GiB RAM, probably mem=16G limits it a little too less to prevent the crash 100% reliably, while mem=15G would do. I don't know how exactly those limits are applied, and at which point exactly the power usage of the RAM raises above the critical point (in case really this is the reason for the crashes), so this is just a guess which can be tested. Edited June 5 by MichaIng 0 Quote
rosbeef Posted June 5 Posted June 5 (edited) @usual user@Efe Çetin it seem to work with Quote DDR_BLOB='rk35/rk3588_ddr_lp4_2112MHz_lp5_2400MHz_v1.16.bin' BL31_BLOB='rk35/rk3588_bl31_v1.45.elf i burned an sd card with the new compiled Armbian-unofficial_24.8.0-trunk_Orangepi5-plus_bookworm_legacy.5.10.160 image . I boot from the sdcard It seems to work as i have no crash copying a 250GB file between a usb ssd and the mvne. I'm trying now memtester 31G. ... ... trying mlock ...Killed BUT not crash trying memtester 30G is working ... ... and it does not crash. @Efe Çetin Is it possible to make the official release working for the 32Gb version ? Edited June 5 by rosbeef 3 Quote
ArtP Posted June 20 Posted June 20 I ran into OPI5+ 32GB stability issues as well. It would throw kernel panic under heavy load. It would throw kernel panic consistently when I did my favourite test: copy the eMMC (or /dev/nvme0n1) to /dev/null using dd. First 4-5 GB would copy, and then - kernel panic. So... I reached out to the manufacturer. Luckily, i bought my board via their Amazon store front. Ultimately, their latest image offered updated u-boot with a new DDR driver. Once I reflashed the SPI flash, my test started working. It is working with the orange pi image and with Armbian image sitting on my /dev/nvme0n1. No crashes so far. 0 Quote
Spo0lsh Posted June 20 Posted June 20 (edited) Hello, I found the solution (for me) about this issue. I hope it will resolve in armbian community too: Remove data on SPI flash chip sudo su - devicesize=$(blockdev --getsz /dev/mtdblock0) dd if=/dev/zero of=/dev/mtdblock0 bs=1M count=$devicesize status=progress && sync Download working uboot image wget https://github.com/si0ls/u-boot-orangepi5/releases/download/v1.0/u-boot-v2024.04-orangepi5-plus-spi.bin Write the u-boot file to SPI flash dd if=u-boot-v2024.04-orangepi5-plus-spi.bin of=/dev/mtdblock0 bs=1M status=progress && sync Reboot the system reboot Enjoy! Some tests (still in progress!): root@orangepi5-plus-nvme:~# memtester 30G 1 memtester version 4.6.0 (64-bit) Copyright (C) 2001-2020 Charles Cazabon. Licensed under the GNU General Public License version 2 (only). pagesize is 4096 pagesizemask is 0xfffffffffffff000 want 30720MB (32212254720 bytes) got 30720MB (32212254720 bytes), trying mlock ...locked. Loop 1/1: Stuck Address : ok Random Value : \ What I use: Orange PI 5 Plus 32GB RAM version Original raspberry pi 4 power supply (USB-C 4A) NVME 128GB M.2 2230 And I tested couple of Armbian images (just execute memtester 30G 1 and wait ~2 minutes for crash ... and when nothing happend then change nvme with another system): # md5sum Armbian_24.* e5d74477e3f976ece5f85e8261491b43 Armbian_24.5.1_Orangepi5-plus_bookworm_vendor_6.1.43_minimal.img ec5c88f73a50ae583c6b07684b28a394 Armbian_24.8.0-trunk.123_Orangepi5-plus_bookworm_edge_6.8.12_minimal.img e29bc71aa4e1d9d4af358faf7ebe81b6 Armbian_24.8.0-trunk.123_Orangepi5-plus_bookworm_vendor_6.1.43_minimal.img Source link : https://github.com/si0ls/u-boot-orangepi5 Edited June 20 by Spo0lsh Fix information 1 Quote
Eddie Posted June 21 Posted June 21 (edited) @Spo0lsh It helped! At least with memtester. Thank you very much! KVM, I'm coming. Edited June 21 by Eddie 0 Quote
ArtP Posted June 21 Posted June 21 (edited) @Spo0lsh This makes sense!. This looks like an equivalent of the "vendor-provided fix" I got from Orange Pi folks. Re-flash SPi with current, good u-boot, and all of sudden things work again. It seems that the DDR driver included with u-boot might be the culprit. I originally had V1.08 20220617 and it was unstable. With the update, I have DDR d5483af87d cym 23/11/23-16:15:24,fwver: v1.15 and things are looking good. The commit message in rockchip-linux/rkbin (place where si0ls/u-boot-orangepi5 gets its blobs from) says "rk3588: ddr: update ddrbin to v1.16". So, chances are, the u-boot there has a DDR driver v1.16 Armbian latest 24.5.1 bookworm image seems to contain u-boot with DDR driver version V1.08 20220617. So, if this gets flashed in SPI, Orange pi5 plus (32 GB version) will become unstable. In hindsite, this is what happened to me. Edited June 21 by ArtP 0 Quote
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.