s_frit Posted February 4, 2019 Posted February 4, 2019 This report applies to NanoPi-4 RK3399 boards (NanoPC T4, NanoPi M4, NanoPi NEO4) running Armbian Bionic (Friendlyarm-based legacy 4.4.y kernel). In other words, boards targeted by Armbian rk3399.conf and Friendlyarm kernel but *NOT* baords using rockchip64.conf and Ayufan kernel. There is a bug in the friendlyarm linux 4.4 version of /scripts/package/builddeb that causes the linux-headers package to be incomplete. Attempts to build out-of-tree kernel modules to fail with errors of the form "fatal error: ../../arm/include/asm/opcodes.h: No such file or directory". For example here is the output trying to build a simple kernel module from this tutorial: Spoiler $ make make -C /lib/modules/4.4.172-rk3399/build M=/home/ross/Documents/lkm_example modules make[1]: Entering directory '/usr/src/linux-headers-4.4.172-rk3399' CC [M] /home/ross/Documents/lkm_example/lkm_example.o In file included from ./arch/arm64/include/asm/sysreg.h:25:0, from ./arch/arm64/include/asm/cputype.h:99, from ./arch/arm64/include/asm/cachetype.h:19, from ./arch/arm64/include/asm/cache.h:19, from include/linux/cache.h:5, from include/linux/printk.h:8, from include/linux/kernel.h:13, from include/linux/list.h:8, from include/linux/module.h:9, from /home/ross/Documents/lkm_example/lkm_example.c:2: ./arch/arm64/include/asm/opcodes.h:5:10: fatal error: ../../arm/include/asm/opcodes.h: No such file or directory #include <../../arm/include/asm/opcodes.h> ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. scripts/Makefile.build:283: recipe for target '/home/ross/Documents/lkm_example/lkm_example.o' failed make[2]: *** [/home/ross/Documents/lkm_example/lkm_example.o] Error 1 Makefile:1499: recipe for target '_module_/home/ross/Documents/lkm_example' failed make[1]: *** [_module_/home/ross/Documents/lkm_example] Error 2 make[1]: Leaving directory '/usr/src/linux-headers-4.4.172-rk3399' Makefile:3: recipe for target 'all' failed make: *** [all] Error 2 @hbmaennchen encountered the same problem when compiling an existing module here: https://forum.armbian.com/topic/8693-headers-44162/ This is a known bug that has been encountered in other kernel header distributions, for example here and here. The fix was applied to mainline in 2016 and is a one line change to the scripts/package/builddeb script. The fix was posted to LKML in March 2016 and was committed to mainline here: https://github.com/torvalds/linux/commit/962475ac2f9662fe69a9f86220ab31bbbf3911d2#diff-03db9657e7716d9ca27a586068430b37 Please could this patch be applied to the Armbian build process? I have no experience with Armbian build but if you need me to generate the patch in the correct format and submit a pull request I can do so, just let me know that you'd like me to do it. Update: my system information NanoPC T4 Ubuntu 18.04.1 LTS bionic 4.4.172-rk3399 header package installed: linux-headers-rk3399 Update: steps to reproduce Using a NanoPi-4 RK3399 board (NanoPC T4, NanoPi M4, NanoPi NEO4) running Armbian Bionic (Friendlyarm-based legacy 4.4.y kernel) First install the linux-headers package: $ sudo apt-get install linux-headers-rk3399 Then, place the attached files lkm_example.c and Makefile into a directory (e.g. named lkm_example). cd to that directory, and type `make`. You should then get the error in the spoiler above. The expected behavior is for the make to succeed without error (which should happen if you are running any other board with the appropriate linux-headers-... package, e.g. the same board with the dev kernel, and linux-headers-dev-rk3399 package installed. Makefile lkm_example.c
s_frit Posted February 4, 2019 Author Posted February 4, 2019 For completeness, the issue is only present in friendlyarm 4.4 repo: https://github.com/friendlyarm/kernel-rockchip/blob/nanopi4-linux-v4.4.y/scripts/package/builddeb#L325 Which is the source for the default kernel for 'rk3399' family boards such as nanopc-t4 (https://github.com/armbian/build/blob/master/config/sources/rk3399.conf). It looks like the fix has already been applied to both the 4.4 and mainline ayufan-rock64 repositories (which are the source for 'rockchip64' boards): https://github.com/ayufan-rock64/linux-kernel/blob/release-4.4/scripts/package/builddeb#L325 https://github.com/ayufan-rock64/linux-mainline-kernel/blob/master/scripts/package/builddeb#L158 (not sure whether Armbian is sourcing the release-4.4 branch though).
s_frit Posted February 4, 2019 Author Posted February 4, 2019 I have made progress by applying the attached patch and rebuilding linux-headers-rk3399.deb I then installed the deb manually with `sudo dpkg -i ./linux-headers-rk3399_5.74_arm64.deb` then `sudo apt-get install -f` However now I'm getting a different error when compiling test kernel module attached above "/bin/sh: 1: ./scripts/recordmcount: not found": Spoiler $ make make -C /lib/modules/4.4.172-rk3399/build M=/home/ross/Documents/lkm_example modules make[1]: Entering directory '/usr/src/linux-headers-4.4.172-rk3399' CC [M] /home/ross/Documents/lkm_example/lkm_example.o /bin/sh: 1: ./scripts/recordmcount: not found scripts/Makefile.build:283: recipe for target '/home/ross/Documents/lkm_example/lkm_example.o' failed make[2]: *** [/home/ross/Documents/lkm_example/lkm_example.o] Error 127 make[2]: *** Deleting file '/home/ross/Documents/lkm_example/lkm_example.o' Makefile:1499: recipe for target '_module_/home/ross/Documents/lkm_example' failed make[1]: *** [_module_/home/ross/Documents/lkm_example] Error 2 make[1]: Leaving directory '/usr/src/linux-headers-4.4.172-rk3399' Makefile:3: recipe for target 'all' failed make: *** [all] Error 2 I have found some references to needing to "make scripts" in the headers directory, however I've been unable to find a way to do that successfully. Any hints would be appreciated. Update: when running `sudo make scripts` in /usr/src/linux-headers-4.4.172-rk3399 I get the following error Spoiler nanopct4:/usr/src/linux-headers-4.4.172-rk3399$ sudo make scripts CHK scripts/mod/devicetable-offsets.h HOSTCC scripts/selinux/genheaders/genheaders scripts/selinux/genheaders/genheaders.c:13:10: fatal error: classmap.h: No such file or directory #include "classmap.h" ^~~~~~~~~~~~ compilation terminated. scripts/Makefile.host:91: recipe for target 'scripts/selinux/genheaders/genheaders' failed make[3]: *** [scripts/selinux/genheaders/genheaders] Error 1 scripts/Makefile.build:484: recipe for target 'scripts/selinux/genheaders' failed make[2]: *** [scripts/selinux/genheaders] Error 2 scripts/Makefile.build:484: recipe for target 'scripts/selinux' failed make[1]: *** [scripts/selinux] Error 2 Makefile:590: recipe for target 'scripts' failed make: *** [scripts] Error 2 This appears to be a similar issue to here: https://forum.armbian.com/topic/7253-rock64-cannot-install-wireguard-module-on-ubuntu-1804/ I will try the suggested fix once I have had some sleep. fix-linux-headers-rk3399-pkg.patch
Igor Posted February 4, 2019 Posted February 4, 2019 2 hours ago, s_frit said: I will try the suggested fix once I have had some sleep. Thanks! https://github.com/armbian/build/commit/4348bf68a002d30c9cb4d166f52bc5ce4057a73b
s_frit Posted February 5, 2019 Author Posted February 5, 2019 Following on from my previous status, where `make scripts` was failing, I found that selinux security related headers were absent from the linux-headers package. The best (most minimal) fix for this I found was to modify builddeb to add the security include files to the linux-headers package. The patch is an additional one line change (in addition to the fix-linux-headers-rk3399-pkg.patch above): +(cd $srctree; find security/*/include -type f) >> "$objtree/debian/hdrsrcfiles" With this change, both `make scripts` and my test kernel module compile successfully. And the test kernel module loads/unloads correctly. I think this is the good change because it only adds include files, and this addition is already done for sunxi-dev and rockchip-dev (maybe elsewhere too?) e.g.: https://github.com/armbian/build/blob/master/patch/kernel/sunxi-dev/general-packaging-4.20-dev.patch#L153 https://github.com/armbian/build/blob/master/patch/kernel/rockchip-dev/general-packaging-4.20-dev.patch#L153 @Igor I have attached a new patch file called fix-linux-headers-rk3399-pkg-v2.patch this patch includes the change from the previous patch. Please use the new -v2 patch to overwrite the existing fix-linux-headers-rk3399-pkg.patch in Armbian build. Once this propagates to the package repository I will re-run my tests. For future reference, here is how I performed the test: First, make sure that the existing linux-headers package is uninstalled: Spoiler nanopct4:/$ cd /usr/src nanopct4:/usr/src$ sudo apt list --installed | grep linux-headers nanopct4:/usr/src$ sudo apt-get remove linux-headers-rk3399 nanopct4:/usr/src$ sudo rm -rf linux-headers-4.4.172-rk3399/ Then install the new (working) linux-headers package that I built using Armbian build (once Igor updates the repo, just install linux-headers in the usual way). Spoiler nanopct4:/usr/src$ sudo dpkg -i ~/linux-headers-rk3399_5.74_arm64-v2.deb nanopct4:/usr/src$ sudo apt-get install -f Run `make scripts` as per https://docs.armbian.com/User-Guide_Advanced-Features/#how-to-build-a-wireless-driver Spoiler nanopct4:/usr/src$ cd linux-headers-4.4.172-rk3399/ nanopct4:/usr/src/linux-headers-4.4.172-rk3399$ sudo make scripts `make scripts` should complete with no error. Then running `make` in lkm_example directory (described in my first post) should work. And you can follow the instructions in the tutorial to load/unload it from the kernel. fix-linux-headers-rk3399-pkg-v2.patch
Igor Posted February 5, 2019 Posted February 5, 2019 2 hours ago, s_frit said: Once this propagates to the package repository I will re-run my tests. make scripts finishes without error now: root@nanopct4:/usr/src/linux-headers-4.4.172-rk3399# make scripts CHK scripts/mod/devicetable-offsets.h
s_frit Posted February 13, 2019 Author Posted February 13, 2019 On 2/5/2019 at 10:55 AM, Igor said: make scripts finishes without error now: root@nanopct4:/usr/src/linux-headers-4.4.172-rk3399# make scripts CHK scripts/mod/devicetable-offsets.h I just upgraded to the 4.4.174 headers from stable (with apt upgrade from the official armbian apt repo, not using side-loaded packages), and now there is a new problem with `make scripts`: ross@nanopct4:/usr/src/linux-headers-4.4.174-rk3399$ sudo make scripts scripts/kconfig/conf --silentoldconfig Kconfig net/Kconfig:83: can't open file "net/wireguard/Kconfig" scripts/kconfig/Makefile:37: recipe for target 'silentoldconfig' failed make[2]: *** [silentoldconfig] Error 1 Makefile:576: recipe for target 'silentoldconfig' failed make[1]: *** [silentoldconfig] Error 2 make: *** No rule to make target 'include/config/auto.conf', needed by 'scripts'. Stop. @Igor Is it possible that this is fixed with the [Wireguard fix] commit 53fcb9852 ?
s_frit Posted February 13, 2019 Author Posted February 13, 2019 Workaround for latest `make scripts` issue is described here:
Lope Posted March 3, 2019 Posted March 3, 2019 To reproduce issue git clone --depth 1 https://github.com/gordboy/rtl8812au.git cd rtl8812au make -j 4 #you'll see that it fails to do modpost with the stock 4.19.20 kernel. # #Previously I had compiled a custom kernel for 4.19.13. #I downgraded to my 4.19.13 custom kernel and was able to complete the above without any issue. #Kernel headers seem to be broken. #If you're going to run the next step, you should backup your armbian 8812au.ko kernel module. #next step I couldn't get to sudo make install
Lope Posted March 10, 2019 Posted March 10, 2019 My (directly) above post got moved here from the Armbian H5 forum Please note: 1. I experienced the issue on a NanoPi Neo2 v1.0 (H5) board. 2. I did not experience the issue on a Rockchip SoC. Nowhere does this thread mention Neo2 or H5, so I'm mentioning this here so it can at least come up in a search. Maybe my above post got moved here in error?
Igor Posted March 10, 2019 Posted March 10, 2019 11 hours ago, Lope said: Maybe my above post got moved here in error? No, the problem is shared. (workaround, mentioned few posts back, should work)
Recommended Posts