torz77

I can confirm that my changes have now been merged into the upstream linux_openfd repo, so I have deleted my fork. Unfortunately I am unable to edit my instructions, but hopefully it should be clear from the instructions that if my fork is unavailable, then the original repo should be used (alternatively, if a mod could edit the post to remove the reference to my fork, to make the instructions clearer, that would also be appreciated).

Firefox is called firefox-esr in Debian (Extended Support Release)

PR Has been merged. I rebuilt on commit 1cf20837d27a3c09d93395f075cc305c81b4663f and things look good. Build instructions remain the same as my last post on this /_\ _ _ _ __ | |__(_)__ _ _ _ ___ _ _ _ _ ___ / _|/ _(_)__(_)__ _| | / _ \| '_| ' \| '_ \ / _` | ' \___| || | ' \/ _ \ _| _| / _| / _` | | /_/ \_\_| |_|_|_|_.__/_\__,_|_||_| \_,_|_||_\___/_| |_| |_\__|_\__,_|_| v25.11 rolling for Odroid C1 running Armbian Linux 6.12.49-current-meson Packages: Debian stable (trixie) Support: DIY (custom image) IPv4: (LAN) 192.168.1.95, 10.8.0.1, 10.10.0.1 (WAN) ww.xx.yy.zz IPv6: 0000:1111:2222:3333:444:555:6666:7777 Performance: Load: 2% Uptime: 23:36 Memory usage: 8% of 986M CPU temp: 49°C Usage of /: 15% of 6.9G

The easiest way to do this is through a chroot (or perhaps systemd-nspawn) but how practical this is will be dependent on your end goals. I do this all the time.

At the risk of speaking too soon, I think I have found the issue. I need to revert some changes, and do some final testing, but hopefully a PR will be inbound to fix this in the next couple of days.

I have had a bit of time to do some testing (re: MMC disconnects) This bug does not appear in kernel 6.6 A version of this bug appears in 6.10, but without the full trace. Output at the point of disconnect is: [ 1082.901830] EXT4-fs error (device mmcblk0p2): __ext4_get_inode_loc_noinmem:4512: inode #25776: block 1769: comm cron: unable to read itable block [ 1082.914822] EXT4-fs (mmcblk0p2): Remounting filesystem read-only or [ 8235.333023] EXT4-fs (mmcblk0p2): shut down requested (2) It is worth noting that the device is noticeably more sluggish on this kernel (not measured, and may or may not be related) On kernel 6.12 The bug manifests with a full trace: [ 1956.200890] I/O error, dev mmcblk0, sector 9337968 op 0x1:(WRITE) flags 0xc800 phys_seg 1 prio class 2 [ 1965.163111] rcu: INFO: rcu_sched self-detected stall on CPU [ 1965.165764] rcu: 0-....: (57 ticks this GP) idle=9194/1/0x40000004 softirq=13082/13083 fqs=48 [ 1965.173801] rcu: (t=2142 jiffies g=20165 q=305 ncpus=4) [ 1965.179160] CPU: 0 UID: 0 PID: 78 Comm: kworker/0:3 Not tainted 6.12.49-current-meson #1 [ 1965.179720] Hardware name: Amlogic Meson platform [ 1965.179991] Workqueue: events dbs_work_handler [ 1965.180753] PC is at __netif_receive_skb_core.constprop.0+0x4/0x10a4 [ 1965.181411] LR is at __netif_receive_skb_list_core+0xf0/0x1f4 [ 1965.181948] pc : [<c10f55bc>] lr : [<c10f674c>] psr: 80030113 [ 1965.182280] sp : f0801d14 ip : 00000000 fp : c4000000 [ 1965.182583] r10: c211a108 r9 : 00000000 r8 : c4000000 [ 1965.182874] r7 : f0801d48 r6 : c400148c r5 : c211a108 r4 : c4e09900 [ 1965.183225] r3 : c2abbd80 r2 : f0801d44 r1 : 00000000 r0 : f0801d40 [ 1965.183585] Flags: Nzcv IRQs on FIQs on Mode SVC_32 ISA ARM Segment none [ 1965.184017] Control: 10c5387d Table: 05ad004a DAC: 00000051 [ 1965.184260] Call trace: [ 1965.184552] __netif_receive_skb_core.constprop.0 from 0xc4003640 [ 1965.262719] sched: DL replenish lagged too much [ 1990.096714] mmc0: card 0002 removed [ 1990.100368] Aborting journal on device mmcblk0p2-8. [ 1990.100590] JBD2: I/O error when updating journal superblock for mmcblk0p2-8. [ 1990.102743] ------------[ cut here ]------------ [ 1990.111672] WARNING: CPU: 0 PID: 131 at drivers/mmc/host/meson-mx-sdio.c:446 meson_mx_mmc_irq_thread+0x168/0x174 [ 1990.121734] Modules linked in: zram zsmalloc binfmt_misc snd_soc_meson_gx_sound_card snd_soc_meson_aiu snd_soc_hdmi_codec snd_soc_meson_card_utils snd_soc_meson_codec_glue snd_soc_core evdev snd_pcm snd_timer meson_ir snd rc_core soundcore meson_saradc thermal_generic_adc cfg80211 rfkill efi_pstore pstore [ 1990.148744] CPU: 0 UID: 0 PID: 131 Comm: irq/35-c1108c20 Not tainted 6.12.49-current-meson #1 [ 1990.148779] Hardware name: Amlogic Meson platform [ 1990.148791] Call trace: [ 1990.148807] unwind_backtrace from show_stack+0x10/0x14 [ 1990.148865] show_stack from dump_stack_lvl+0x54/0x68 [ 1990.148921] dump_stack_lvl from __warn+0x80/0x114 [ 1990.148980] __warn from warn_slowpath_fmt+0x184/0x18c [ 1990.149030] warn_slowpath_fmt from meson_mx_mmc_irq_thread+0x168/0x174 [ 1990.149088] meson_mx_mmc_irq_thread from irq_thread_fn+0x1c/0x7c [ 1990.149150] irq_thread_fn from irq_thread+0x138/0x21c [ 1990.149207] irq_thread from kthread+0xdc/0xf8 [ 1990.149270] kthread from ret_from_fork+0x14/0x28 [ 1990.149314] Exception stack(0xf106dfb0 to 0xf106dff8) [ 1990.149339] dfa0: 00000000 00000000 00000000 00000000 [ 1990.149365] dfc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1990.149388] dfe0: 00000000 00000000 00000000 00000000 00000013 00000000 [ 1990.149420] ---[ end trace 0000000000000000 ]--- [ 1990.574529] EXT4-fs (mmcblk0p2): shut down requested (2) [ 1991.000606] mmc0: new high speed SDHC card at address 0002 [ 1991.001627] mmcblk0: mmc0:0002 00000 7.32 GiB [ 1991.006695] mmcblk0: p1 p2 [ 2041.955601] EXT4-fs warning (device mmcblk0p2): dx_probe:823: inode #593: lblock 0: comm bash: error -5 reading directory block [ 2041.961894] EXT4-fs warning (device mmcblk0p2): dx_probe:823: inode #593: lblock 0: comm bash: error -5 reading directory block [ 2041.974376] EXT4-fs warning (device mmcblk0p2): dx_probe:823: inode #593: lblock 0: comm bash: error -5 reading directory block If I were to speculate it looks like: initial bug introduced kernel 6.7 to 6.10 Bug was identified and the "fix"/"work around" has lead to the more verbose failure somewhere in 6.11 - 6.12 I went for the low hanging fruit first and can confirm the driver meson-mx-sdio.c is unchanged across all versions. I have also compared the DTBs: 6.6 -> 6.10 Significant changes 6.10 -> 6.12 Identical The changes looks to be the USB fix. I suspect that the issue is not with the DTB files, as the significant changes will be down to shifting phandle references by +1, I have now completely ruled out changes to the DTB. I loaded up the 6.12 kernel with the 6.6 DTB and hit the bug fairly quickly (wish I'd done this to start with, to be honest!) When I next get some time I will start diffing changes to the mmc/core and clk/meson drivers between 6.6 -> 6.10. If anyone else wants to have a crack at this in the meantime, based on the above information, then please, have at it!

So I looked into this, and unfortunately, no dice. I am happy to investigate further, if you so wish, but I'm happy enough with the way it works currently (auto patching the dtb after a kernel update) so am unlikely to go exploring further on my own. To let you know what I did: copied the dtbo to /boot/dtb/amlogic/overlay/meson-gxl-s905w-tx3-mini-openvfd.dtbo Added to /boot/extlinux/extlinux.conf: fdtoverlays /dtb/amlogic/overlay/meson-gxl-s905w-tx3-mini-openvfd.dtbo (I also tried with /boot prepended to the path as well, but same issue) note: I assume FDTOVERLAYS is a typo from the armbianEnv.txt days, so I went with the lowercase as that is standard for extlinux The device would not come up on the network. I plugged in an HDMI monitor and found that it would hang at: ethernet end0: renamed from eth0 Which is.... odd. I suspect one of 2 things: My entry in extlinux.conf is incorrect (entirely plausible, I'm struggling to find documentation, I've just gone on what you've said, and also changed the capitalisation!) U-Boot overlays are applied differently than fdtoverlay in Linux. It’s possible U-Boot leaves something unaligned or adds an unexpected property, and the kernel hits a corner case. I realise that the above is noway near enough information, however, connecting directly to serial is not realistic at this stage, but I can attach a netconsole if needs be, as we are reaching the kernel, and provide further output. Like I say though, I will only do this if you think there is value in it. Whilst my method might not be the preferred way, it does work, at least.

I'm not sure what this adds? I saw a user struggling to get it working, I managed to get it working, so thought I would give my steps that they may help other people. I am happy to improve my instructions based on feedback from users like @SteeMan above, but I was not specifically looking for help. If my tutorial and responses are in anyway not wanted here, then feel free to delete them.

Ok, this I was not aware of. When I get a second I will try it out. I'm not sure how long editing remains open on my post, but if it works, and if I can, I will go back and edit

I mean that creating the dts and executing: sudo armbian-add-overlay openvfd.dts Results in: grep: /boot/boot.cmd: No such file or directory Overlays are not supported on Meson64 based boards. I started to investigate, but realised it was a massive rabbit hole, and what with the note in the documentation saying: I decided that directly applying the overlay to the DTB and running an /etc/kernel/postinst.d/ script following a kernel update would be my path of least resistance. If I have overlooked something, I would happily update the instructions as, clearly, using an overlay would be far more preferential. EDIT: I should add, that I also tried compiling and adding the overlay manually, and adding an armbianEnv.txt for boot, but the overlay was not loaded.

<off topic: If someone wants to tell me how to "reply" to a post on these forums, that would be appreciated!> Sadly, Meson64 does not have DTB Overlays enabled. That would, of course, be my preference. I have, however, updated my instructions to create an overlay and apply it directly to the DTB (rather than manually editing the DTB) and pointed users in the direction of postint.d if they wish to automate this (out of scope to provide specific instructions here). It should now be easier to create an overlay rather than directly apply to DTB if the user's box supports this. Unfortunately, I did not see this until I was well underway getting OpenVFD working. The box is now in production, so it is not straightforward taking it out and experimenting with it. However, having more than one way to achieve an end goal is no bad thing imho.

I've just been through this process myself. Those instructions are not close to working, and I'm not sure they ever worked. I have documented how I got the LCD display working on the T3-Mini here:

This was originally meant as a reply to a user having problems enabling openVFD on a Tannix T3-Mini, a device I happen to own. I have recently been through this journey myself, and having searched the forums, I cannot find a recent topic on how to build this for Armbian, so have decided to make a new post that may be of use to some. Hold on to your hat, because this is going to be long. Caveats These instructions are, specifically, for the Tannix TX3-Mini. However, with a bit of fiddling, the general approach should work for any supported TV Box. I have added notes where you will need to look to edit a different file for your specific device There are many, many variants of the TX3-Mini out there. What works for me, may not work for you. Do not expect any help or support from me, I am just posting this as a courtesy for how I got this working... your mileage may vary. I am not going to troubleshoot anyone's issues These instructions are quite verbose, as they may also help users of other TV Boxes to get their displays working. It also may not. Like I say, I am not here to be tech support, but we can all agree not having a display stuck on "boot" is a nice thing to have As this is a kernel module it will most likely stop working after each kernel update. You will probably want to create a DKMS to rebuild the module whenever you download a new kernel. This is outside of scope here. Use Google. At the end of this, if all goes well, you will have a display showing the current time. If you want to do more with the display then this is outside of scope and you will need to look elsewhere. However, this link is useful for how to trigger the icons: https://github.com/arthur-liberman/linux_openvfd/blob/master/led_control.txt (note: only items 1 to 6 are valid for the tx3-mini) A lot of this can be done in a chroot, but the actual building of the kernel module itself must be done on the target device. To simplify things all of these instructions are to be executed on the device itself. If you want to do this in a chroot, then knock yourself out, but you are on your own. My setup At the time of writing, these instructions are confirmed working for the 7 Segment display and all icons on: Tannix T3-Mini S905w with 2GB RAM Armbian 25.11 Kernel 6.12.48-current-meson64 Debian stable (trixie) (13) Instructions Note: Every code block here is meant to be pasted and executed in one go, even the multi-line blocks We will work from the home folder to keep things simple. Don't worry, there will be no clutter as we will remove files we no longer require as we go cd ~ Device Tree Blob The first thing we are going to want to do is enable kernel support for openvfd in our DTB. Normally I'd do this with an overlay, but this does not appear to be enabled on the aml-s9xx-box image, so we will apply an overlay to the DTB directly: Install the device tree compiler: sudo apt install -y device-tree-compiler --no-install-recommends Back up the existing DTB (if anything goes wrong you can always just restore the backed up DTB) : Note: If your device is not a Tanix T3-Mini, then you will want to amend the following to point to the actual DTB you are using (you can find this in '/boot/extlinux/extlinux.conf') sudo cp /boot/dtb/amlogic/meson-gxl-s905w-tx3-mini.dtb /boot/dtb/amlogic/meson-gxl-s905w-tx3-mini.dtb.orig Create the overlay source code: cat << EOF > ~/openvfd.dts /dts-v1/; /plugin/; / { fragment@0 { target-path = "/"; __overlay__ { openvfd { compatible = "open,vfd"; dev_name = "openvfd"; status = "okay"; }; }; }; }; EOF Compile the overlay: dtc -@ -I dts -O dtb -o ~/openvfd.dtbo ~/openvfd.dts Merge the overlay into your DTB: Note: If your device is not a Tanix T3-Mini, then you will want to amend the following to point to the actual DTB you are using sudo fdtoverlay -i /boot/dtb/amlogic/meson-gxl-s905w-tx3-mini.dtb -o /boot/dtb/amlogic/meson-gxl-s905w-tx3-mini.dtb ~/openvfd.dtbo Delete the overlay source: rm ~/openvfd.dts [Optional] Delete the compiled overlay: If your build is static (that is, you will never pull an updated DTB through apt) then you can also delete the compiled .dtbo overlay file. I prefer to keep this around, as you can just re-patch the new DTB with the "sudo fdtoverlay ..." command above. It is also possible to automate the update of a newly installed DTB file by creating a postinst.d script, but that is outside of the scope of this document. Google is your friend. rm ~/openvfd.dtbo Reboot so when we load the module later, our device knows what to do with it sudo reboot now Once your device has been rebooted, you can confirm that your change has been applied correctly with the following command: dtc -I fs -O dts /proc/device-tree | grep -A3 openvfd Again, this will generate a lot of warnings! This is normal. At the end of the warnings you should see the openvfd entry that you added to your DTS in the earlier step. If you do not, then you have not edited the file correctly, and you should go back and try again. OpenVFD Config file We need to create a configuration file which tells the OpenVFD module which GPIO pins are connected to the LCD display. We put this in the /etc folder as this is where we should be storing system configuration files for *deb based systems The contents of this file were extracted from https://github.com/arthur-liberman/vfd-configurations so if you are using a different device, you must replace the following config with the relevant one from the link. If you are having issues with your config not working, direct them to the repo owner, not me. I do not know your device or what may be wrong. Note: I remove the final functions='usb colon eth wifi' line as whilst the driver works fine with it included, it generates errors/warnings, which I would rather not see, and it appears to serve no purpose for Armbian Execute the following to generate the config for the TX3-Mini Note: If your device is not a Tanix T3-Mini do not execute the following. Instead, find your config at https://github.com/arthur-liberman/vfd-configurations and save it as /etc/openvfd.conf sudo bash -c "cat << 'EOF' > /etc/openvfd.conf vfd_gpio_clk='0,76,0' vfd_gpio_dat='0,75,0' vfd_gpio_stb='1,4,0' vfd_chars='4,3,2,1,0' vfd_dot_bits='0,1,3,2,4,5,6' vfd_display_type='0x01,0x00,0x00,0x00' EOF" Build the Kernel Module Now for the nitty gritty, we need to build the kernel module. The first thing we need is the kernel headers. Note: the headers version must match your installed kernel version exactly. Do not try installing the headers for a different kernel version. You will run into issues If you are on a standard image, or your kernel has been upgraded since you built your image, this is straightforward: sudo apt install linux-headers-$(uname -r) However, if you have built the image yourself, and you have not upgraded your kernel, then most likely the version available from the apt repository will not be compatible and your build may fail or the driver may not work at all. In these instances, you will need to go back to your build system and add the following switch to your ./compile.sh command: INSTALL_HEADERS=yes Install the required build tools sudo apt install -y git build-essential --no-install-recommends Clone the openvfd repo. At the time of writing the openvfd repo is not compatible with later Linux kernels. I have raised a PR against the repo to enable support, however it has not yet been accepted. If/when it is accepted I will be deleting my fork of the repo, but in the meantime, you can clone my fork with: git clone https://github.com/torzdf/linux_openvfd.git ~/linux_openvfd If the above does not work, it is because I have deleted my fork as the changes have been merged, and I am unable to come back and edit this post. If this is the case then run the following: Note: DO NOT run the next line, if the above git clone worked git clone https://github.com/arthur-liberman/linux_openvfd.git ~/linux_openvfd Enter the driver folder of the cloned repo cd ~/linux_openvfd/driver Create a Makefile. The provided Makefile will not work, so we need to replace it with our own: cat << 'EOF' > ./Makefile ifeq ($(KERNELRELEASE),) PWD = $(shell pwd) KERNELDIR = /lib/modules/`uname -r`/build modules: $(MAKE) -C $(KERNELDIR) M=$(PWD) modules modules_install: $(MAKE) -C $(KERNELDIR) M=$(PWD) modules_install clean: rm -rf *.o *.ko .tmp_versions *.mod.c modules.order Module.symvers ssd253x-ts.* else obj-m := openvfd.o openvfd-objs += protocols/i2c_sw.o openvfd-objs += protocols/i2c_hw.o openvfd-objs += protocols/spi_sw.o openvfd-objs += controllers/dummy.o openvfd-objs += controllers/seg7_ctrl.o openvfd-objs += controllers/fd628.o openvfd-objs += controllers/fd650.o openvfd-objs += controllers/hd44780.o openvfd-objs += controllers/gfx_mono_ctrl.o openvfd-objs += controllers/ssd1306.o openvfd-objs += controllers/pcd8544.o openvfd-objs += controllers/il3829.o openvfd-objs += openvfd_drv.o endif EOF Compile the kernel module: make -j$(nproc) Install the kernel module: sudo make modules_install Update the kernel modules: sudo depmod -a Create the helper service Next we need to compile and install the helper service Enter the folder that contains the source code for the helper service: cd ~/linux_openvfd Build the helper service: make OpenVFDService Make the helper service executable: chmod +x OpenVFDService Install the helper service: sudo cp OpenVFDService /usr/bin/ Clean up We have built everything we need from the OpenVFD repo, so we can get rid of the source code Go back to our home folder and delete the source code cd ~ && sudo rm -r linux_openvfd systemd Service file The final step. We need to create a service file that will load the kernel module, launch the helper service, and enable it on boot Create the systemd service file: note: If you prefer a 12 hour clock rather than a 24 hour clock, edit the 'Environment="OPTS=-24h"' line to 'Environment="OPTS=-12h"' sudo bash -c 'cat << '\''EOF'\'' > /etc/systemd/system/openvfd.service [Unit] Description=openvfd Wants=network-online.target [Service] Type=simple Environment="OPTS=-24h" ExecStartPre=/usr/bin/sh -c ". /etc/openvfd.conf; /usr/sbin/modprobe openvfd vfd_gpio_clk=$vfd_gpio_clk vfd_gpio_dat=$vfd_gpio_dat vfd_gpio_stb=$vfd_gpio_stb vfd_chars=$vfd_chars vfd_dot_bits=$vfd_dot_bits vfd_display_type=$vfd_display_type;" ExecStart=/usr/bin/OpenVFDService $OPTS & ExecStop=/usr/bin/killall OpenVFDService ExecStopPost=-/usr/sbin/rmmod openvfd [Install] WantedBy=multi-user.target EOF' Reload the systemd daemon: sudo systemctl daemon-reload Start the openvfd service: sudo systemctl start openvfd.service At this point your LCD should now be showing the time. If it is not, you can check for errors with: sudo systemctl status openvfd.service Enable the service at boot: sudo systemctl enable openvfd.service And that's it. If all has gone well, you now have a working LCD Display for your TV Box running a recent Armbian build

Certainly worth a look. It is also probably worth comparing the Armbian commit from when I last had it working until now, but it will need to be when I have a bit more time, and me remembering just when I first started seeing these issues.

An update. I suspect that the issue (or at least 1 issue) is a regression in the handling of MMC (SD card driver in the kernel). I suspect this is a regression as this issue did not hit me on earlier builds. I set up a netconsole, and received the following stack trace: [ 216.236507] I/O error, dev mmcblk0, sector 849688 op 0x0:(READ) flags 0x84700 phys_seg 1 prio class 2 [ 216.524566] EXT4-fs warning (device mmcblk0p2): ext4_end_bio:353: I/O error 10 writing to inode 131108 starting block 323201) [ 216.534718] EXT4-fs warning (device mmcblk0p2): ext4_end_bio:353: I/O error 10 writing to inode 131109 starting block 323202) [ 216.544920] EXT4-fs warning (device mmcblk0p2): ext4_end_bio:353: I/O error 10 writing to inode 131110 starting block 323203) [ 216.560591] EXT4-fs warning (device mmcblk0p2): ext4_end_bio:353: I/O error 10 writing to inode 131111 starting block 323204) [ 216.575914] EXT4-fs (mmcblk0p2): failed to convert unwritten extents to written extents -- potential data loss! (inode 131108, error -5) [ 216.587187] Buffer I/O error on device mmcblk0p2, logical block 270977 [ 216.601434] EXT4-fs (mmcblk0p2): failed to convert unwritten extents to written extents -- potential data loss! (inode 131109, error -5) [ 216.601564] ------------[ cut here ]------------ [ 216.608348] Buffer I/O error on device mmcblk0p2, logical block 270978 [ 216.608383] WARNING: CPU: 0 PID: 129 at drivers/mmc/host/meson-mx-sdio.c:446 meson_mx_mmc_irq_thread+0x168/0x174 [ 216.608467] EXT4-fs (mmcblk0p2): failed to convert unwritten extents to written extents -- potential data loss! (inode 131110, error -5) [ 216.608485] Buffer I/O error on device mmcblk0p2, logical block 270979 [ 216.612894] Modules linked in: rpcsec_gss_krb5 [ 216.619667] EXT4-fs (mmcblk0p2): failed to convert unwritten extents to written extents -- potential data loss! (inode 131111, error -5) [ 216.629626] auth_rpcgss [ 216.629674] Buffer I/O error on device mmcblk0p2, logical block 270980 [ 216.629700] zram [ 216.632164] Aborting journal on device mmcblk0p2-8. [ 216.641941] zsmalloc [ 216.642033] JBD2: I/O error when updating journal superblock for mmcblk0p2-8. [ 216.648457] binfmt_misc [ 216.676718] EXT4-fs error (device mmcblk0p2): __ext4_find_entry:1639: inode #598: comm dpkg-reconfigur: reading directory lblock 0 [ 216.680959] evdev [ 216.681075] EXT4-fs (mmcblk0p2): I/O error while writing superblock [ 216.683223] snd_soc_meson_aiu snd_soc_meson_gx_sound_card snd_soc_hdmi_codec snd_soc_meson_card_utils snd_soc_meson_codec_glue snd_soc_core snd_pcm meson_ir snd_timer rc_core [ 216.683268] EXT4-fs (mmcblk0p2): Remounting filesystem read-only [ 216.683278] thermal_generic_adc snd meson_saradc soundcore cfg80211 rfkill efi_pstore pstore netconsole [ 216.744185] CPU: 0 UID: 0 PID: 129 Comm: irq/35-c1108c20 Not tainted 6.12.47-current-meson #3 [ 216.744207] Hardware name: Amlogic Meson platform [ 216.744212] Call trace: [ 216.744225] unwind_backtrace from show_stack+0x10/0x14 [ 216.744259] show_stack from dump_stack_lvl+0x50/0x64 [ 216.744275] dump_stack_lvl from __warn+0x80/0x11c [ 216.744294] __warn from warn_slowpath_fmt+0x17c/0x188 [ 216.744311] warn_slowpath_fmt from meson_mx_mmc_irq_thread+0x168/0x174 [ 216.744337] meson_mx_mmc_irq_thread from irq_thread_fn+0x1c/0x7c [ 216.744364] irq_thread_fn from irq_thread+0x134/0x224 [ 216.744378] irq_thread from kthread+0xe0/0xfc [ 216.744396] kthread from ret_from_fork+0x14/0x28 [ 216.744411] Exception stack(0xf1075fb0 to 0xf1075ff8) [ 216.744420] 5fa0: 00000000 00000000 00000000 00000000 [ 216.744428] 5fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 216.744435] 5fe0: 00000000 00000000 00000000 00000000 00000013 00000000 [ 216.744458] ---[ end trace 0000000000000000 ]--- [ 217.163204] mmc0: new high speed SDHC card at address 0002 [ 217.163907] mmcblk0: mmc0:0002 Which suggests the SD card was connected and disconnected. I have already ruled out faulty SD card (happens on any SD card I use, as well as a thorough stress test on the current card prior to use). Further research tells me "The Amlogic Meson8b (S805) SoC’s MMC controller has known quirks." I switched to USB rootfs (external HDD), and I still get the same failure, but as it is just the boot partition being unmounted and remounted the system remains up. Ideally I would propose a patch to fix this, but I am not a kernel developer, and the changes between 3.10 and 6.12 are huge (as to be expected) so identifying what is required and where is non-trivial for me. For now, I am sticking with rootfs on USB as my interim fix, but I am going to keep netconsole up and monitor over the next couple of weeks to see if we hit another issue (when I tested this in the past, I was also getting failures with rootfs on an external USB stick, so I am not convinced I am completely out of the woods yet).