Minimum steps to build a kernel module for latest Armbian release

[Blade Doyle]

Hi Armbians,

 

(If this question is already asked/answered please help me find the link.)

 

I need the "ceph" and "rbd" kernel modules for my orange and banana pi.  I am starting with the most recent images released: Armbian_4.5_xxxx_Debian_jessie_4.2.3.

 

What are the minimum required steps to build the additional kernel modules?  Where is the source tree for that kernel build (4.3.3-sunxi)?

 

I would prefer to build directly on the PI (as opposed to x-compile) but either is ok.

 

I tried a few things, including git clone of Igors "lib" project with build scripts.  Runnig that pulled a kernel source tree: "sources/linux-vanilla/v4.3.3" which I am able to use to build my kernel modules, but they fail to load with:  "modprobe: ERROR: could not insert 'rbd': Exec format error".

 

Please enlighten me.

Blade.

[Igor]

I think the best / most simple way is to change to your custom kernel and install it over stock. Alter revision number to 9999 that it won't be overwritten with stock while apt-get update / upgrade. 

 

The purpose of "lib" tools are exactly that.

 

This kernel module copy method usually doesn't work between different kernels.

[Blade Doyle]

Thanks, that worked.

 

Looks like the versions were not exactly the same after all, which is likely why the modules were not loading.

 

"Unpacking linux-image-next-sunxi (4.82) over (4.81) ..."

 

So now I have what I need!

 

root@node4:~# lsmod
Module                  Size  Used by
ceph                  157660  0
rbd                    48507  0
libceph               129427  2 rbd,ceph
 

My banana pi pro, and 2 orange pi plus are now able to join the 4 raspberry pi2 in my 'mixed fruit pi Kubernetes cluster'.  Awesome.

 

Thanks for all your hard work!

Blade.

[Igor]

Cool!

 

Just one add regarding kernel name, since you haven't alter the name. Next apt-get upgrade will overwritten your kernel ... one option is altering revision/name and using a custom one, the other is to share your kernel config and i'll merge it to our default.

[Blade Doyle]

Thanks Igor.  I understand about the name, and will resolve as you suggest.

[Blade Doyle]

For the record, the config changes to generate modules needed to for Ceph and RBD filesystems are:

 

% diff .config.old .config
1408c1408,1410
< # CONFIG_CEPH_LIB is not set
---
> CONFIG_CEPH_LIB=m
> # CONFIG_CEPH_LIB_PRETTYDEBUG is not set
> # CONFIG_CEPH_LIB_USE_DNS_RESOLVER is not set
1574c1576
< # CONFIG_BLK_DEV_RBD is not set
---
> CONFIG_BLK_DEV_RBD=m
4549c4551,4552
< # CONFIG_CEPH_FS is not set
---
> CONFIG_CEPH_FS=m
> # CONFIG_CEPH_FS_POSIX_ACL is not set

 

And, to get Docker to run "group-enable=memory" needs to be added to /boot/boot.cmd

[Igor]

https://github.com/igorpecovnik/lib/commit/9fcc1cbea207922aa88f0e8307e543290f8e3032

 

You can leave your custom naming as is since next kernel will have all this inside.

 

Hmm, what happen if we leave this on by default?

 

group-enable=memory

[Blade Doyle]

My bad, it should be "cgroup-enable=memory".  (I missed the initial "c" in my previous post)

 

Also, my additional research inidicates that "swapaccount=1" should also be added to Debian boot params for Docker.

 

For some reason, for Debian systems, even though its configued in the kernel it is still necessary to enable it using a boot option. 

 

 

Ref:

 

https://github.com/docker/docker/issues/396

 

The docker community seems to feel its a bug in Debian, so adding those to your standard distribution is probably the right thing to do.

[Igor]

https://github.com/igorpecovnik/lib/commit/7be413aca5f7727272b979a3412c1c1970d8bdb5

 

OK, tnx

[Dmitry]

Hello All, I've tried to setup Ceph on orange, but without any success (Unable to locate package ceph-osd, ceph-mon etc  during running ceph-deploy install). So I suppose I need those kernel modules too? Could you please advise me on how to build and install them??

My gratitude will have no limits!

[Igor]

10 hours ago, Dmitry said:

I've tried to setup Ceph on orange, but without any success (Unable to locate package ceph-osd, ceph-mon etc  during running ceph-deploy install)

Since install can't locate some packages means you have missed something in their install instructions. We provide stock Debian / Ubuntu and headers, which are needed for possible module installation are already in place - no need to fetch them. 

[rcampos]

Can we also get this applied to other boards by default, I'm using Odroid-N2 and have to recompile the kernel to get the RBD module, would be great to get it as standard.

# CONFIG_BLK_DEV_RBD is not set

 

Thanks

[martinayotte]

1 hour ago, rcampos said:

have to recompile the kernel to get the RBD module, would be great to get it as standard.

Feel free to add CONFIG_BLK_DEV_RBD=m in every config/kernel/ files and then submit a PR.

[rcampos]

2 hours ago, martinayotte said:

Feel free to add CONFIG_BLK_DEV_RBD=m in every config/kernel/ files and then submit a PR.

Here's the pr for Ceph + RBD as a module by default on meson, meson64, odroidn2

https://github.com/armbian/build/pull/1665