Building Kernel does not build modules (*.ko)

[Tony3]

I must be doing something stupid.

After having switched the branch "Multimedia" as M, no module are generated at all and I end up with error like:

🔨]   ERROR: modpost: "dma_contiguous_default_area" [drivers/media/common/videobuf2/videobuf2-cma-sg.ko] undefined!
[🔨]   ERROR: modpost: "cma_release" [drivers/media/common/videobuf2/videobuf2-cma-sg.ko] undefined!
[🔨]   ERROR: modpost: "cma_alloc" [drivers/media/common/videobuf2/videobuf2-cma-sg.ko] undefined!
 

Reading documentation to buld armbian, it only mention docker. I have been (last year) generating kernel on the machine itself (the orange pi 5 plus), this is not supported?

 

thanks.

 

 

 

[SteeMan]

Please post you full build logs

[Igor]

1 hour ago, Tony3 said:

this is not supported?

It is. We even use a few of those machines to host build runners. Docker documentation is largely outdated. Now its does everything automatically, but you need to install Docker itself IIRC.

[Tony3]

Find attached the log

log-build-2b5bb7f0-f9a9-4c7c-9690-8e8040c3e6f2.log log-build-2b5bb7f0-f9a9-4c7c-9690-8e8040c3e6f2.log.ans summary-build-2b5bb7f0-f9a9-4c7c-9690-8e8040c3e6f2.md

[Tony3]

Note: later, I deselected the usb gadget, but still get the same error.

[SteeMan]

So in looking at the diff of your kernel config there is a lot different from base armbian.  I'm assuming the image build fine if you don't modify the kernel config file?

 

What are you trying to accomplish?  Why are you editing config files?

[Tony3]

Really? Being too different is not the goal.

I need the armbian config, and only switch media support branch as module, in order to compile the drivers based on linuxtv media_build.

 

I downloaded armbian by following instructions I think, running the build from the machine having a boot/config and would like to use that boot config.

How can I use the boot/config of my machine as a baseline and regenerate an armbian kernel?

 

Edited June 9 by Tony3
Added more background info

[SteeMan]

At that point you are doing kernel development.  There is no reason that the kernel source code being used by Armbian (the vendor branch) supports the features you are enabling.  It is very likely that patches from mainline linux will need to be backported to the vendor kernel branch that you are trying to build.  This isn't an Armbian issue, but something you would need to do.  You are essentially adding features (by enabling kernel feature that Armbian doesn't curently support or test).  If you want to go down this path, you should enable items that you need in your kernel config one by one and see which specific features are failing.

Again, if you diff your kernel config with the one Armbian uses there are many differences.

[Tony3]

Thanks for your reply and support. First of all, when I download and build without changes, the armbian kernel build works fine.

Then when I switch media support branch to M I get the error. I am not adding any new feature, only switching to M media support. Does that M add features compared to armbian? I thought it would only make a part of armbian config a module, thus not adding any new feature?

 

Something else I noticed is that if after switching to M , if I switch media support  back to Y, then I still get the error, where I thought I would be back in a no error compilation. There is obviously a concept I do not understand there, any clue?

[Igor]

1 hour ago, Tony3 said:

First of all, when I download and build without changes, the armbian kernel build works fine.

(Armbian) Linux kernel is a common work, product which we maintain in certain areas so you can run Linux on those devices better. Defaults are also where our best effort "warranty" ends - this can be checked with automation, manual testing of *all features or even fixing them is impossible. Linux kernel is a complex machinery and a work of tens of thousands of people. Nobody really knows if feature you are trying to get working in the kernel builds or not and if it works at the end. Ready (def)configs are usually made with a reason that you start with a known-to-work configuration, proceed slowly and fix things on your way. Similar as we do in some areas. Keeping this code build-able and in good shape is already on the very extreme expense end. This is open source and all work is public, value can be reused by everyone, sadly also by competitors that contributes nothing, hardware vendors also don't just help out of humanitarian reasons because we help you using their products.

 

1 hour ago, Tony3 said:

There is obviously a concept I do not understand there, any clue?

 

Code quality in Linux kernel is different. Especially if we are talking about vendor kernels. There, only HW features are maintained, nothing else, so this difference is just extreme. But also in mainline Linux, code is usually build-able & usable only until maintainers (largely amateurs) are doing their hard work. Dead and orphaned parts of code are regularly removed, but this functionally deprecation is a process that takes time, a lot of code is broken for long time, decisions are arbitrary. We don't do that as we don't have capacity for patching this megalomaniac chunk of code. We leave things as is. Only upstream Linux does that in small %, here and there, and even they are considered small and under-powered. Most of people in Linux (FOSS) land are sadly not maintaining critical infrastructure such as kernel, but doing some "me too" spin, copy of copy, Linux distribution. Shipping someone code is significantly cheaper that developing and maintaining it.

 

When you start doing what you do, things can quickly become messy and unpredictable. Armbian framework, infrastructure and this community can't do miracles, but provides added value. It doesn't make things worse and that is already a great achievement.

[Tony3]

Thanks for your explanation and I understand it isn't simple.

I would like to try a simple step by step approach.

 

I downloaded the armbian as explained in documentation, what do I need to do to start recompiling exactly the version I have on my orange pi 5 plus?

 

I compile on the machine itself, where/when do I copy my /boot/config in the downloaded build directory?

 

Thanks Again 

[going]

10 часов назад, Tony3 сказал:

I would like to try a simple step by step approach.

Your new configuration has been exported:

(119) INFO: Exporting new kernel config [ /home/eric/BuildKernel/build/output/config/linux-rk35xx-vendor.config ]

The first step is to compare it with the old default

cd /home/eric/BuildKernel/build/
diff -u config/kernel/linux-rk35xx-vendor.config output/config/linux-rk35xx-vendor.config

Did that tell you something?

Mutual dependencies in Kconfig may be inaccurate.

 

    MODPOST Module.symvers
   ERROR: modpost: module usb_f_uvc uses symbol dma_buf_attach from namespace DMA_BUF, but does not import it.
   ERROR: modpost: module usb_f_uvc uses symbol dma_buf_map_attachment from namespace DMA_BUF, but does not import it.
   ERROR: modpost: module usb_f_uvc uses symbol dma_buf_unmap_attachment from namespace DMA_BUF, but does not import it.
   ERROR: modpost: module usb_f_uvc uses symbol dma_buf_detach from namespace DMA_BUF, but does not import it.
   ERROR: modpost: "dma_contiguous_default_area" [drivers/media/common/videobuf2/videobuf2-cma-sg.ko] undefined!
   ERROR: modpost: "cma_release" [drivers/media/common/videobuf2/videobuf2-cma-sg.ko] undefined!
   ERROR: modpost: "cma_alloc" [drivers/media/common/videobuf2/videobuf2-cma-sg.ko] undefined!
   make[2]: *** [scripts/Makefile.modpost:126: Module.symvers] Error 1

I suspect that this driver cannot be built as a module because it does not export the necessary symbols.

But in the Kconfig file, the ability to put the letter M is present.
Unfortunately, this is a common mistake of programmers.
You will have to fix the kernel code if you really want to build the driver as a module or leave everything as it is.

[going]

11 часов назад, Tony3 сказал:

I would like to try a simple step by step approach.

 

The kernel compilation script contains "brute force" code that does:

--> (46) COMMAND: ./scripts/config --enable EXPERT
--> (46) COMMAND: ./scripts/config --enable CONFIG_MODULE_COMPRESS_NONE
--> (46) COMMAND: ./scripts/config --set-str CONFIG_LOCALVERSION ""
--> (46) COMMAND: ./scripts/config --enable CONFIG_IKCONFIG
--> (46) COMMAND: ./scripts/config --enable CONFIG_IKCONFIG_PROC
--> (46) COMMAND: ./scripts/config --enable CONFIG_GPIO_SYSFS

 

11 часов назад, Tony3 сказал:

do I copy my /boot/config in the

Using your configuration file may not work correctly due to the above.

First, compare it with the default configuration file.

[Tony3]

First of all thanks to have helped me on that matter.

I found out what the issue was. When switching some option to M, some code had to be patched as it sounds like it was not expecting to become module or calling a module.

The general root cause is that some C code is missing instruction like "MODULE_IMPORT_NS(DMA_BUF);", to use module symbols of another module or like "EXPORT_SYMBOL_GPL(dma_contiguous_default_area);" to make some symbols available to other modules.

After making the whole "Media support" branch as a module, I had to make the attached patchs and apply them to my linux kernel 6.1 V24.05 by copying them in the directory build/userpatches/kernel/rk35xx-vendor-6.1

 

After doing that all went fine, I got the new Armbian Kernel built, and was able to compile the tbs drivers against it by following instruction there https://github.com/tbsdtv/linux_media/wiki

 

All drivers working fine with tvheadend and using Kodi as a viewer (with hardware acceleration)

 

At the end, I am impressed that it worked that well, thanks to Armbian to make the building of a kernel so easy.

cma.patch contiguous.patch f_uvc.patch

[going]

2 часа назад, Tony3 сказал:

After doing that all went fine, I got the new Armbian Kernel built, and was able to compile the tbs drivers against it by following instruction there https://github.com/tbsdtv/linux_media/wiki

 

All drivers working fine with tvheadend and using Kodi as a viewer (with hardware acceleration)

Did I understand correctly?
The ultimate goal of this work was to support TBS in the kernel?

 

 

[Tony3]

Yes it was to support TBS satellite tuners, surprised?

[going]

4 минуты назад, Tony3 сказал:

Yes it was to support TBS satellite tuners, surprised?

I'm just not aware of this topic (satellite TV).

 

@Igor @jock The changes in the published patches seem insignificant. Does it make sense to add them to the build system?

[jock]

Can't really get a clue about the issue. I mean, in the mainline kernel cma_alloc seems to be used in a non-core driver, altough it is very seldom used: https://elixir.bootlin.com/linux/latest/A/ident/cma_alloc so I guess that's not something client drivers should use.

 

Going back to rockchip 5.10 vendor kernel source code laying on my notebook I see:

 

paolo@armbian-build:~/rk3528/linux-rockchip/mm$ rgrep -IH 'cma_' | grep EXPORT
cma.c:EXPORT_SYMBOL_GPL(cma_get_name);
cma.c:EXPORT_SYMBOL_GPL(cma_alloc);
cma.c:EXPORT_SYMBOL_GPL(cma_release);
cma.c:EXPORT_SYMBOL_GPL(cma_used_pages);
cma.c:EXPORT_SYMBOL_GPL(cma_for_each_area);

 

But those EXPORTs are not present in the mainline 5.10 kernel...

 

This old LWN article perhaps explain the rationale:

Quote

[...] In fact, device drivers should never need to call the CMA API directly, since instead of bus addresses and kernel mappings it operates on pages and page frame numbers (PFNs), and provides no mechanism for maintaining cache coherency. [...]

 

and things like dma_alloc_coherent() should be called instead: there are hundreds of references to that, and it's far easier call too.

Perhaps the driver should be fixed to use the proper API, as exporting those symbols actually seems a violation to me.

 

I don't have a rational opinion on including those small patches in the armbian vendor kernel; vendor kernel often host many of those shortcuts; unfortunately shortcuts are one of the reasons they are also hard to maintain.

 

The ideal would fix the driver.

 

edit: it puzzles me that the kernel linked by @Tony3 does not contain those EXPORTs either: https://github.com/tbsdtv/linux_media/blob/latest/mm/cma.c

[Tony3]

@jock. Yes it is weird, but i did not used the tbs one.

Concerning the update of the TBS drivers, this is a bit out of my control, but I could as well raise the issue there and see what happens.

Thanks to be proactive as I agree that twists are not long term solutions.

[going]

15 часов назад, jock сказал:

I don't have a rational opinion on including those small patches in the armbian vendor kernel; vendor kernel often host many of those shortcuts; unfortunately shortcuts are one of the reasons they are also hard to maintain.

Thanks for the explanation, Paolo.

 

@Tony3 Perhaps a more accurate name of the topic (problem) could gather people here who are aware of the issue.

And the answers would be more qualified.

[MaxT]

I've played with TBS drivers a while ago for tinkerboard https://forum.armbian.com/topic/10464-how-to-add-a-kernel-module-for-usb-dvb-t-tuner/?do=findComment&comment=79667.

At that time I thought that there were smth in vendor's code of 4.4 in dvb-core that prevented TBS drivers loading, though later I've found that TBS is also patching dvb-core and some other modules like CEC, frontends, etc.

So may be solution would be to build all modules required for TBS drivers (stating from dvb-core? ) and use them instead of mainline/vendor or extract TBS drivers and other TBS changes from its repo as patches series to apply in armbian.

Though I don't know how to compare the whole trees to extract TBS drivers as patches. Any hints will be appreciated.

[Tony3]

Yes feasible but would require some time.

As the method above with the patches works fine with my tbs cards on pi5 and pi5 plus, i will first start to inform tbs not to call the function above directly. Will see how this works!