Trouble Building and Loading the `rbd` Module with Armbian Kernel

[LoneExile]

Hello, Armbian community,

 

I've been trying to build the Armbian kernel for my board "Orange Pi 5" with the rbd (RADOS Block Device) module enabled. I followed the standard build procedure using Armbian's build tools, and I explicitly enabled the rbd support during the menuconfig step. Despite doing this, after completing the build and installation processes, I'm unable to load the rbd module.

When I attempt to load the module with modprobe rbd, I get the following error:

 

modprobe: FATAL: Module rbd not found in directory /lib/modules/5.10.160-rk35xx

 

Here are the steps I followed:

1. Cloned the Armbian Build Repository.

2. Started the kernel configuration using:

./compile.sh

 

1. In Menuconfig, I navigated to Device Drivers > Block devices and enabled Rados block device (RBD) support as a module.

2. Continued with the build process and installed the newly built kernel.

3. Booted the board and tried to load the module, which resulted in the error.

 

Has anyone faced a similar issue or can you provide guidance on what I might be doing wrong or missing? Any help or suggestions would be much appreciated!

I really appreciate any help you can provide.

 

 

 

[schwar3kat]

In the screenshot, you are not root, so all modprobe commands require sudo privileges.
Try:
sudo modprobe rbd

[LoneExile]

 

Thank you for your response.

I understand the importance of the appropriate privileges. I did indeed try the modprobe command with sudo earlier, but I still received the same error:

 

 

To ensure I was running the most up-to-date software, I also executed `sudo apt update && sudo apt upgrade -y`, but unfortunately, the issue persists.

[Werner]

Check kernel config in /boot if the mentioned module is marked as M there. From my guts feel I'd say I hasn't even compiled in.

[schwar3kat]

I don't have your board, so I can't test, and I'm not familiar with Rados block device, but I will build it and see if I can see the module.
 

[LoneExile]

Thank you, @Werner. I've already left my house, but I'll quickly check the kernel config in `/boot` as soon as I get home. Just for my understanding, if the mentioned module is not marked as M there, the next step would be to re-compile the kernel, correct? If it is marked as M, do you have any further recommendations?

And @schwar3kat, I appreciate you taking the time to look into this. Yes, as seen in the previous screenshot, compiling the kernel took around 2 hours for me. It's kind of you to offer your assistance despite not having my board or familiarity with the Rados block device.

[schwar3kat]

Just started - and I can see it configured in the Makefile.  "obj-$(CONFIG_BLK_DEV_RBD)     += rbd.o"
Halfway through - and it's built the module object.  "rdb.o"
Completed - and I can see the file "rdb.o" in the image in "/lib/modules/5.10.160-rk35xx/kernel/drivers/block" which is correct. (Do you have this in your image?).

/boot/config-5.10.160-legacy-rk35xx contains  "CONFIG_BLK_DEV_RBD=m" as expected.

Looks okay as far as i understand, but I'm no expert.  Did you save your new configuration after selection and exits?


Your second build on the same clone should be faster because of caching.

[LoneExile]

@schwar3kat Thank you for pointing out the build config file. I overlooked enabling `CONFIG_BLK_DEV_RBD`.

It was an oversight on my part, and I appreciate your help. Thanks again for taking the time.

[Dexogen]

I apologize, but I still haven’t figured out how to correctly build an image with RBD support. Could you share detailed instructions?

 

1. Cloned and started build script.

git clone --depth=1 --branch=main https://github.com/armbian/build
./compile.sh

2. In Menuconfig Device Drivers > Block devices > Rados block device (RBD) set <M>.
3. Saved config as .conf and continued build process.
4. In the end, I received the image Armbian_23.11.0-trunk_Orangepi5-plus_bookworm_legacy_5.10.160.img from which I downloaded, but there is no module in it.

 

I read the thread several times but still don’t understand how the topicstarter solved the problem.

[LoneExile]

@Dexogen I followed the same instructions as you did to build the image, but I encountered a snag. After running `apt update && apt upgrade`, the `rbd` module seemed to get overwritten somehow. I stopped investigating further when I found that using `ubuntu jammy edge` worked for me. I hope this helps!

[Dexogen]

@LoneExile Thanks for the answer! Unfortunately, I need the Debian version of the distribution because I'm installing Proxmox. I discovered that the default build bookworm_edge_6.6.0-rc1 contains the RBD module. But here I encountered the surprising behavior of the Orange Pi 5 Plus board: now it boots not every time. I literally have to switch the power supply several times until the OS boots. This does not happen on the bookworm_legacy_5.10.160 kernel version. I have three identical Oragne Pis with different power supplies and this problem is reproduced on the bookworm_edge_6.6.0-rc1 kernel.  But this is a question for a another thread. Nevertheless thank you!

[Dexogen]

By some miracle I built a legacy kernel 5.10.160 with RBD support. I can't say exactly what exactly helped, but I did the following:
 

1. I literally added output/config/linux-* and config/kernel/linux-* to every file:

CONFIG_BLK_DEV_RBD=m

2. Added to ./config/boards/orangepi5-plus.conf

MODULES="rbd"

3. Repeated previous build instructions.
 

And on this kernel the OPie5+ boots without problems every time.

[rinseaid]

Thanks, @Dexogen, this was helpful for me. I found that the MODULES line wasn't needed, and specifically the linux-rk35xx-legacy kernel config file is used for these boards.

 

Here were the full steps I followed to successfully build an Armbian image with ceph RBD & cephfs support for a PVE arm64 installation. I built on an Ubuntu Jammy qemu VM running on an x86 Proxmox host, and ran as a normal user with sudo privileges as per the Armbian build docs. I needed to ensure the machine could reach external DNS (I think Cloudflare but I just allowed all outbound UDP/TCP 53) to build successfully - I only allow certain DNS servers in my network. You also need more than 10GB free space for the build to complete successfully. Ignore the warnings regarding shallow build.

git clone --depth=1 --branch=main https://github.com/armbian/build
cd build
sed -i 's/.*CONFIG_BLK_DEV_RBD.*/CONFIG_BLK_DEV_RBD=m/' config/kernel/linux-rk35xx-legacy.config
sed -i 's/.*CONFIG_CEPH_FS.*/CONFIG_CEPH_FS=m/' config/kernel/linux-rk35xx-legacy.config
./compile.sh build BOARD=orangepi5-plus BRANCH=legacy BUILD_DESKTOP=no BUILD_MINIMAL=no KERNEL_CONFIGURE=no RELEASE=bookworm

 

I also have some Rock 5b boards, you can simply replace 'orangepi5-plus' with 'rock-5b' in the above commands to build for that board.

 

It's also currently necessary to downgrade the bundled edk2 firmware for rk3588 to succesfully boot VMs. I used the following commands:

apt download pve-edk2-firmware=3.20220526-1 && dpkg -i pve-edk2-firmware_3.20220526-1_all.deb && rm pve-edk2-firmware*.deb

echo 'Package: pve-edk2-firmware
Pin: version 3.20220526-1
Pin-Priority: 999' > /etc/apt/preferences.d/pve-edk2-firmware

 

Edited October 28, 2023 by rinseaid