Matthijs Kooijman

For anyone else looking to patch their kernel through userpatches - it seems that (at least with the updated build system that uses the quite complex lib/tools/patching.py to apply patches) you can just drop a patch in `userpatches/kernel/archive/<platform>-<version>/` and it will be picked up (in my case, it was `userpatches/kernel/archive/sunxi-6.7`). No series file needed. I used a patch generated with `git format-patch`, but it seems that the script supports different formats, even mbox with multiple patches inside.

Right, then the suggestion about this error earlier in the thread was incorrect. I had seen it was wifi-related, but had assumed the same radio is used for wifi and bluetooth, so thought the error might have been related (though I suspected it was just a warning and a second firmware load would succeed, but mentions of it online were a bit confusing). Anyway, thanks for confirming!

I just tested a clean image, installed just `pi-bluetooth` and everything worked right way (hciconfig shows a bluetooth adapter and I can pair devices using gnome-settings right away). In particular, I do not need armbian-firmware-full, nor the firmware symlinks I discussed above. On closer inspection, I suspect that the firmware error message that we were seeing was just a first attempt at loading a raspberrypi4-specific firmware, but it already falls back to a generic firmware. In particular, looking at the OP's dmesg output (which matches my own), I see the following lines (unrelated lines were snipped): [ 9.387030] brcmfmac mmc1:0001:1: Direct firmware load for brcm/brcmfmac43455-sdio.raspberrypi,4-model-b.bin failed with error -2 [ 9.658798] brcmfmac: brcmf_fw_alloc_request: using brcm/brcmfmac43455-sdio for chip BCM4345/6 [ 9.658998] brcmfmac: brcmf_fw_alloc_request: using brcm/brcmfmac43455-sdio for chip BCM4345/6 [ 9.679093] brcmfmac: brcmf_c_preinit_dcmds: Firmware: BCM4345/6 wl0: Nov 1 2021 00:37:25 version 7.45.241 (1a2f2fa CY) FWID 01-703fd60 I think this means it first tries brcm/brcmfmac43455-sdio.raspberrypi,4-model-b.bin, that fails, then it tries brcm/brcmfmac43455-sdio which succeeds. Also, I did not need to install bluetooth packages using armbian-config (those probably add support for other bluetooth stuff, basic keyboard and mouse already worked). Concluding: Installing `pi-bluetooth` is enough to make bluetooth work in a pi4, so that should probably be installed by default in this image (it is also a tiny image).

I suspect I do not have this package - it does seem like that package would indeed work, since it calls hciattach using this service and this script. I had seen mention of this package before, but I suspected it was something specific to Raspbian, or maybe Ubuntu itself, hadn't realized it would be available on armbian as well. I'll test to see if installing it helps and if so, we should probably install it by default (in the image, or in the armbian-config bluetooth set of packages maybe). I checked the kernel config, and the rpi image (bcm2711) already has those modules builtin, so that apparently does not work here (maybe that's a mainline feature to automatically do the hciattach, as you mention it does work automatically on mainline apparently).

Turns out running hciattach works: $ sudo hciattach /dev/ttyAMA0 bcm43xx [sudo] password for matthijs: bcm43xx_init Flash firmware /lib/firmware/brcm/BCM4345C0_003.001.025.0162.0000_Generic_UART_37_4MHz_wlbga_ref_iLNA_iTR_eLG.hcd Set Controller UART speed to 3000000 bit/s Device setup complete $ sudo hciconfig hci0: Type: Primary Bus: UART BD Address: E4:5F:01:3D:8E:21 ACL MTU: 1021:8 SCO MTU: 64:1 UP RUNNING RX bytes:928 acl:0 sco:0 events:69 errors:0 TX bytes:5491 acl:0 sco:0 commands:69 errors:0 After that, I can connect to my bluetooth keyboard using gnome settings as normal. What I did: - Install bluetooth with armbian-config - Install armbian-firmware-full - Create firmware symlinks - Run the hciattach command I'm not sure if all of these are really needed, I'll try with a clean image to be sure. If I can find the time, I'll also see if we can make this work out of the box somehow, by running the hciconfig command at startup (this pullrequest for sunxi adds a systemd service for this, I was thinking of a udev rule maybe). I also found this pullrequest for sunxi which makes some kernel modules builtin and then claims hciattach is not needed at all. Maybe also see how Raspbian does this...

I'm running into the same issue on my rpi4b (jammy gnome version), same messages in dmesg as DownUnder has, and no hci shown by `hciconfig` (so I assume the problem is indeed kernel-level). - Installing `armbian-firmware-full` does not help. It contains files named like the requested firmware, but .txt instead of .bin (which I guess might *also* be needed). These txt files are symlinks (which again link to other files): $ ls -l /lib/firmware/brcm/*raspberrypi,4-model-b* lrwxrwxrwx 1 root root 22 Jul 28 2022 /lib/firmware/brcm/brcmfmac43455-sdio.raspberrypi,4-model-b.txt -> brcmfmac43455-sdio.txt -rw-rw-r-- 1 root root 1883 Feb 17 23:23 /lib/firmware/brcm/brcmfmac43455-sdio.raspberrypi,4-model-b.txt.distrib lrwxrwxrwx 1 root root 22 Jul 28 2022 /lib/firmware/brcm/brcmfmac43456-sdio.raspberrypi,4-model-b.txt -> brcmfmac43456-sdio.txt - I found this thread where someone (for the rpi3) suggests adding rpi symlinks to the generic files, which I adapted for rpi4: sudo ln -s ../cypress/cyfmac43455-sdio.clm_blob brcmfmac43455-sdio.raspberrypi,4-model-b.clm_blob sudo ln -s ../cypress/cyfmac43455-sdio.bin brcmfmac43455-sdio.raspberrypi,4-model-b.bin This fixes the error in dmesg, but hciconfig still shows nothing. $ sudo dmesg|grep brcm [ 1.427393] brcm-pcie fd500000.pcie: host bridge /scb/pcie@7d500000 ranges: [ 1.427443] brcm-pcie fd500000.pcie: No bus range found for /scb/pcie@7d500000, using [bus 00-ff] [ 1.427529] brcm-pcie fd500000.pcie: MEM 0x0600000000..0x063fffffff -> 0x00c0000000 [ 1.427621] brcm-pcie fd500000.pcie: IB MEM 0x0000000000..0x007fffffff -> 0x0400000000 [ 1.492518] brcm-pcie fd500000.pcie: link up, 5.0 GT/s PCIe x1 (SSC) [ 1.492963] brcm-pcie fd500000.pcie: PCI host bridge to bus 0000:00 [ 8.509484] brcmfmac: F1 signature read @0x18000000=0x15264345 [ 8.530822] brcmfmac: brcmf_fw_alloc_request: using brcm/brcmfmac43455-sdio for chip BCM4345/6 [ 8.538523] usbcore: registered new interface driver brcmfmac [ 8.815847] brcmfmac: brcmf_fw_alloc_request: using brcm/brcmfmac43455-sdio for chip BCM4345/6 [ 8.823298] brcmfmac: brcmf_c_preinit_dcmds: Firmware: BCM4345/6 wl0: Nov 1 2021 00:37:25 version 7.45.241 (1a2f2fa CY) FWID 01-703fd60 - I found this post that suggests you need to run hciattach to set things up (I suspect the bluetooth adapter might be attached to serial and this creates a hci device talking to that serial?). I haven't tried this yet, though. To be continued, but I gotta run now, so this post is a bit rough, sorry for that.

Building a git master "cli" image does seem to boot. E.g. with this commandline: ./compile.sh docker BOARD=rockpi-s BRANCH=edge RELEASE=jammy BUILD_DESKTOP=no BUILD_MINIMAL=no KERNEL_ONLY=no KERNEL_CONFIGURE=no and armbian-build master 21186c0577c39699e04949366f094bda54960729, I built Armbian_23.02.0-trunk_Rockpi-s_jammy_edge_6.1.4.img which boots as expected. Building 22.11.2 locally also shows the same problem (does not boot). E.g. with this commandline: ./compile.sh docker BOARD=rockpi-s BRANCH=edge RELEASE=jammy BUILD_DESKTOP=no BUILD_MINIMAL=no KERNEL_ONLY=no KERNEL_CONFIGURE=no REPOSITORY_INSTALL=kernel,u-boot ./compile.sh docker BOARD=rockpi-s BRANCH=edge RELEASE=jammy BUILD_DESKTOP=no BUILD_MINIMAL=no KERNEL_ONLY=no KERNEL_CONFIGURE=no REPOSITORY_INSTALL=kernel,u-boot and armbian-build v22.11 e323753fe7fb26f833dbfeab94351b38fe97550d, I built Armbian_22.11.2_Rockpi-s_jammy_edge_6.0.10.img which does not boot like the official builds. Note that I used REPOSITORY_INSTALL here, since a locally-built kernel failed for some reason (net/bluetooth/hci_sync.c:3442:42: error: ‘HCI_QUIRK_BROKEN_PARK_LINK_STATUS’ undeclared.), which I have not further investigated. Building git master with prebuilt kernel and u-boot does *not* boot: ./compile.sh docker BOARD=rockpi-s BRANCH=edge RELEASE=jammy BUILD_DESKTOP=no BUILD_MINIMAL=no KERNEL_ONLY=no KERNEL_CONFIGURE=no REPOSITORY_INSTALL=kernel,u-boot and armbian-build master 21186c0577c39699e04949366f094bda54960729, I built Armbian_23.02.0-trunk_Rockpi-s_jammy_edge_6.0.10.img which fails to boot again. Note that the kernel is actually different from the self-built master version, which got 6.1.4. This suggests that the problem is in the kernel, and 6.0.10 kernel is broken and 6.1.4 fixed it, except that the problem also occurred with 5.19.16 jammy, but that same kernel worked with bullseye (so at least there is more than just the kernel involved). Doing the same built with just REPOSITORY_INSTALL=kernel (so rebuilding u-boot) also fails, removing REPOSITORY_INSTALL entirely again (so the same built as the first one in this post), things boot again, confirming that the difference is not in something that has been cached along the way. Conclusion: Likely the problem has been fixed in git master. I'll retest the official images once 23.02 is released (if I remember...).

Prompted by your tests, I also did another test, maybe the problem has been fixed since then. IIUC you tested with images built from git master (are those autobuilt and available somewhere? I did not find them at https://www.armbian.com/rockpi-s/ or http://xogium.performanceservers.nl/archive/rockpi-s/nightly/), but I tested released versions only (but now I tested 22.11, before it was 22.08, hoping it would be fixed between those versions already). The images were downloaded two weeks ago or so, so maybe they are not the completely latest versions. Interestingly enough, I discovered that the regular cli image still does not boot (same behaviour and serial output), but the minimal version *does* boot. That's actually surprising, since I would expect both versions to share the same kernel and most of the initrd contents (maybe minimal has a bit less). This image fails to boot: Armbian_22.11.2_Rockpi-s_jammy_edge_6.0.10.img rock-pi-s-fail.txt This image boots Armbian_22.11.2_Rockpi-s_jammy_edge_6.0.10_minimal.img boot rock-pi-s-ok.txt Diff of serial output (up to Staring kernel... line): --- rock-pi-s-fail.txt 2023-01-10 11:40:36.181577309 +0100 +++ rock-pi-s-ok.txt 2023-01-10 11:41:44.916939972 +0100 @@ -9,7 +9,7 @@ Boot1 Release Time: Mar 24 2022 08:28:57, version: 1.36 ROM VER:0x56323030, 19 chip_id:330800,0 -ChipType = 0x13, 481 +ChipType = 0x13, 483 DPLL = 1300 MHz ...nandc_flash_init enter... No.1 FLASH ID:ff ff ff ff ff ff @@ -50,9 +50,9 @@ NeedKHz=40000KHz,clock=650000KHz SdmmcInit=0 0 BootCapSize=0 -UserCapSize=29554MB +UserCapSize=29844MB FwPartOffset=2000 , 0 -StorageInit ok = 27861 +StorageInit ok = 27952 SecureMode = 0 Secure read PBA: 0x4 Secure read PBA: 0x404 @@ -70,7 +70,7 @@ No find bl32.bin Load uboot, ReadLba = 2000 Load OK, addr=0x600000, size=0x994f8 -RunBL31 0x40000 @ 104386 us +RunBL31 0x40000 @ 105251 us ^AINFO: Preloader serial: 0 NOTICE: BL31: v1.3(release):30f1405 NOTICE: BL31: Built : 17:08:28, Sep 23 2019 @@ -87,7 +87,7 @@ INFO: SPSR = 0x3c9 -U-Boot 2022.04-armbian (Dec 23 2022 - 09:48:40 +0000) +U-Boot 2022.04-armbian (Dec 23 2022 - 09:41:50 +0000) Model: Radxa ROCK Pi S DRAM: 254 MiB @@ -105,27 +105,174 @@ mmc1 is current device Scanning mmc 1:1... Found U-Boot script /boot/boot.scr -3045 bytes read in 4 ms (743.2 KiB/s) +3045 bytes read in 6 ms (495.1 KiB/s) ## Executing script at 00500000 Boot script loaded from mmc 1 176 bytes read in 4 ms (43 KiB/s) -23208702 bytes read in 987 ms (22.4 MiB/s) -31273472 bytes read in 1327 ms (22.5 MiB/s) -54638 bytes read in 10 ms (5.2 MiB/s) +16449244 bytes read in 701 ms (22.4 MiB/s) +31273472 bytes read in 1329 ms (22.4 MiB/s) +54638 bytes read in 11 ms (4.7 MiB/s) Failed to load '/boot/dtb/rockchip/overlay/rk3308-fixup.scr' Moving Image from 0x680000 to 0x800000, end=2670000 ## Loading init Ramdisk from Legacy Image at 04000000 ... Image Name: uInitrd Image Type: AArch64 Linux RAMDisk Image (gzip compressed) - Data Size: 23208638 Bytes = 22.1 MiB + Data Size: 16449180 Bytes = 15.7 MiB Load Address: 00000000 Entry Point: 00000000 Verifying Checksum ... OK ## Flattened Device Tree blob at 02800000 Booting using the fdt blob at 0x2800000 - Loading Ramdisk to 0c904000, end 0df262be ... OK + Loading Ramdisk to 0cf76000, end 0df25e9c ... OK ERROR: reserving fdt memory region failed (addr=0 size=0 flags=0) - Loading Device Tree to 000000000c88e000, end 000000000c903fff ... OK + Loading Device Tree to 000000000cf00000, end 000000000cf75fff ... OK Starting kernel ... Surprisingly, the diff is bigger then between the bullseye and jammy builds in my original posts. I'll see if I can build a cli an minimal image from git master myself to see if those boot normally or not.

Increasing verbosity to 7 did not change the output at all. AFAIU that option only increases verbosity of the kernel, not u-boot, right? Then maybe the kernel does not manage to start executing at all maybe? I doublechecked the serial settings in the boot.scr, which indeed point to ttyS0. I also tried setting `earlycon=on` in armbianEnv.txt, but that did not make a difference either...

I just tried the official Jammy image on the Rock Pi S, which does not boot. Looking at serial, it halts after "Starting kernel ...". I've tried redownloading the image, burning it again and on a different SD card. I also tried the bullseye image, which *does* boot (which makes me believe this is not a hardware issue). Anyone with a Rock Pi S that could confirm my observation? This image failed to boot: Armbian_22.08.7_Rockpi-s_jammy_edge_5.19.16.img Serial output: rock-pi-s-fail.txt This image booted ok: Armbian_22.08.7_Rockpi-s_bullseye_edge_5.19.16.img Serial output: rock-pi-s-ok.txt Diff of serial output (up to Staring kernel... line): --- rock-pi-s-fail.txt 2022-11-09 12:40:03.199949846 +0100 +++ rock-pi-s-ok.txt 2022-11-09 12:58:16.937136391 +0100 @@ -52,7 +52,7 @@ BootCapSize=0 UserCapSize=29554MB FwPartOffset=2000 , 0 -StorageInit ok = 27839 +StorageInit ok = 27858 SecureMode = 0 Secure read PBA: 0x4 Secure read PBA: 0x404 @@ -70,7 +70,7 @@ No find bl32.bin Load uboot, ReadLba = 2000 Load OK, addr=0x600000, size=0x9dad0 -RunBL31 0x40000 @ 106200 us +RunBL31 0x40000 @ 106201 us ^AINFO: Preloader serial: 0 NOTICE: BL31: v1.3(release):30f1405 NOTICE: BL31: Built : 17:08:28, Sep 23 2019 @@ -108,8 +108,8 @@ 3045 bytes read in 5 ms (594.7 KiB/s) ## Executing script at 00500000 Boot script loaded from mmc 1 -176 bytes read in 5 ms (34.2 KiB/s) -23219213 bytes read in 986 ms (22.5 MiB/s) +176 bytes read in 4 ms (43 KiB/s) +19531454 bytes read in 831 ms (22.4 MiB/s) 31263232 bytes read in 1328 ms (22.5 MiB/s) 54638 bytes read in 9 ms (5.8 MiB/s) Failed to load '/boot/dtb/rockchip/overlay/rk3308-fixup.scr' @@ -117,15 +117,33 @@ ## Loading init Ramdisk from Legacy Image at 04000000 ... Image Name: uInitrd Image Type: AArch64 Linux RAMDisk Image (gzip compressed) - Data Size: 23219149 Bytes = 22.1 MiB + Data Size: 19531390 Bytes = 18.6 MiB Load Address: 00000000 Entry Point: 00000000 Verifying Checksum ... OK ## Flattened Device Tree blob at 02800000 Booting using the fdt blob at 0x2800000 - Loading Ramdisk to 0c8fd000, end 0df21bcd ... OK + Loading Ramdisk to 0cc81000, end 0df2167e ... OK ERROR: reserving fdt memory region failed (addr=0 size=0 flags=0) - Loading Device Tree to 000000000c887000, end 000000000c8fcfff ... OK + Loading Device Tree to 000000000cc0b000, end 000000000cc80fff ... OK Starting kernel ...

Your post is about the Rock Pi S, but you link to the Rock 5?

I won't have time for that, too many other projects already... (including some other projects I rescued like this :-p). I will be maintaining my own fork a bit, but only for my particular and very specific usecase, so going beyond that and properly maintaining the project (including development and/or accepting contributions) is going to take up a lot more time...

I needed rpi-clone support for Armbian as well, so I implemented at least basic support at https://github.com/billw2/rpi-clone/pull/140 Testing welcome (probably best to just download the script from the pullrequest, since there has not been significant activity in that repo, so I do not expect the PR to be merged anytime soon...)

Hi all, While building a custom image that needs to do some things at the end of the first bootup and reboot, I ran into some issues with armbian-firstrun.service. It currently has Type=simple, which means dependencies will be started when armbian-firstrun is *started*, and there is no clean way to wait until after it has *finished*. Looking to fix this, I noticed some other things that could be improved in this area, such as using systemd's first-boot-complete.target and ConditionFirstBoot to more simply manage first-boot-only services. I'm considering implementing some changes and submitting them in a PR, but before I do that, I'd like to get some feedback on the approach and whether it seems worth investing time into. I considered creating a github issue about this, but given it is sort-of a feature request and the github new issue wizard seems intended to redirect away from github issues, I thought to instead post here. If a github issue seems more appropriate, I'll gladly repost there. In any case, to solve the particular problem I was having (a service that needs to run after the full first boot has completed, including resizing and firstrun script), here's three incremental changes that I would think would make sense (just 1. would be the minimal to solve this problem, 1. + 2. would solve it more generally, and 1. + 2. + .3 would simplify the code a bit maybe). 1. Make armbian-firstrun.service `Type=oneshot`, so you can declare `After=armbian-firstrun.service` and `After=armbian-resize-filesystem.service`. This also means that system boot is delayed until this service is completed, but it does not prevent other boot services to run in parallel, so I would not consider this an issue (if it is, then removing the `Before=getty...` stuff could be considered, which is currently pointless anyway (you will not get a login prompt until armbian-firstrun.service is started, but it will not wait for completion). Note that using `Type=simple` in armbian-firstrun.service originates from https://github.com/armbian/build/commit/ee8d396fa639cd89e08fdd6646c32308f3b25f4f, but there is no rational for this choice. 2. Give armbian-firstrun.service and armbian-resize-filesystem `Wants=first-boot-complete.target` and `Before=first-boot-complete.target` so `first-boot-complete.target` does not fire until `armbian-firstrun` is done, allowing others to just do `After=first-boot-complete.target` without having to explicitly reference specific services. 3. Give armbian-firstrun.service and armbian-resize-filesystem `ConditionFirstBoot=yes` to let systemd ensure they are only called on the first boot. This allows removing the `systemctl disable` calls in these respective scripts as well, but needs the `/etc/machine-id` file to be removed (see https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=900366#20), but that's probably a good idea anyway (to be regenerated on first boot, currently all machines using the same image have the same machine id). One thing (feature or seeming problem maybe) with using ConditionFirstBoot as suggested above, is that systemd ensures that if the first boot is interrupted, the first boot services are ran again on the subsequent boots, until they have all ran completely (to ensure they all had a chance to ran). This is implemented by (roughly) running ConditionFirstBoot services only if `/etc/machine-id` does not exist, and only writing it to disk *after* all `first-boot-complete.target` services have completed (also when they failed, I think). This means that if the boot is interrupted before *all* first-boot services are ran, all of them will be ran again on the next boot and needs these services to be ok with being ran twice (this is also currently the case when the boot is interrupted when either or both services are still running, except that if either service completes, then that particular service will not be retried, but others might be). See also https://man7.org/linux/man-pages/man5/machine-id.5.html As a related optimization, if 1. above is applied I think we could also add a `Before=ssh.service` to armbian-firstrun.service and remove the sshd restart from `armbian-firstrun`, so systemd just delays sshd startup until the keys are generated (but only if armbian-firstrun is active, so on subsequent boots sshd is started as normal). Also, regardless of all of the above, I see that the armbian-firstrun deletes the ssh host keys and then forces regenerating them by calling dpkg-reconfigure. However, it would seem safer to me to actually delete the ssh keys at the end of image generation somewhere (maybe in `post_debootstrap_tweaks()`?), to really rule out the possibility of machines all using the same (publicly known) keys from the image file. So, how do these sound?

I've also been seeing the size mismatch errors during builds and investigated a bit further. I believe the problem is caused by the combination of mirrors being out of sync (which is unavoidable, I do not think it is feasible to sync them all at the same time), and apt.armbian.com redirecting to different mirrors between requests (as Igor also pointed out). I created a github issue with some more details and a suggested fix here: https://github.com/armbian/dl-router/issues/29 I believe the fix for this would be to make apt.armbian.com do consistent redirects (i.e. redirect the same IP to the same mirror all the time, or at least for longer periods of time). @Igor, you said above that "It's a feature, not a bug", are you referring to the unavoidability of mirrors being out of sync, or also the inconsistent redirects? IOW, are you suggesting this is not fixable (or should not be fixed)?