Patch and configuration directory organization

[TonyMac32]

This topic is a response to the growing number of kernels, growing number of boards with... "unique properties", and for the increased complexity of some platofrms (aarch64 and the atf, etc).  I think the coherency of the chosen organizational method is starting to fall apart.

 

So just a thought experiment on my part:

 

Currently, we have a sources directory, and the sources file has not just sources, but also executable script functions in it.  We also do our patches per SoC, mostly, but the organization is by the thing being compiled rather than by the target, making finding/adjusting a bit cumbersome in some cases. 

 

I would propose a directory tree looking something like this:

 

├── RK3288
│   ├── kernel
│   │   ├── default
│   │   ├── dev
│   │   └── next
│   ├── miqi
│   ├── tinkerboard
│   │   ├── kernel
│   │   │   ├── default
│   │   │   ├── dev
│   │   │   └── next
│   │   └── uboot
│   │       ├── default
│   │       ├── dev
│   │       └── next
│   └── uboot
│       ├── default
│       ├── dev
│       └── next
└── S905

In this case the organization is by SoC, then by board.  All patches safe for the entire board population get put in the SoC-level kernel and uboot patch folders, anything unsafe (hardkernel-specific patches, Tinker board specific patches) go into the board-level folders.  To reinforce that we want those folder to be "last resort", we could call them "C2_bugs, Tinker_bugs" etc.

 

For our 64-bit targets, the trusted firmware would likewise have it's directory.  I think it might also make for a lower barrier to entry on committing fixes, I still worry about patching things when I don't have all the boards to test on, this way, as a novice or as someone doing something that I can't test on other hardware, I can drop the patch into the board(s) I have, then they can be evaluated by the other devs on the remainder of the hardware.

 

My particular set of boards benefit this way:

 

• Tinker board specific patches are many, integrating them safely is one of the biggest reasons we lag the vendor kernel, changes to the architecture files, changes to rk3288.dtsi instead of just to rk3288-minarm.dts, changes to various drivers for specific hardware issues, etc etc etc etc.
• Differences in gxbb vs gxl Meson64 devices.  Most of these coexist peacefully, but then you have vendor issues with u-boot of the C2, K2, etc (everyone has their own bootloader blobs)

 

I'd say we organize bootloader scripts similarly, rather than having executable scripts in the sources file we could place an include in the appropriate board folder covering the specifics. For example, if a board ships with a SPI boot flash, we don't need to burn u-boot, only provide the proper configuration of partitions/etc. That might be the only board in the series (an unusually awesome H3 board, for example).

 

Ideally, at that point, adding a new board would require a new board definition file, and then the proper patches folder. 

 

[lanefu]

4 hours ago, TonyMac32 said:

Ideally, at that point, adding a new board would require a new board definition file, and then the proper patches folder. 

 

Tangential thought... adding a script for making scaffolding/templating for a new board would be cool. (board def, folders etc). I suppose it could help assure the proposed structure as well.

 

I'll admit i haven't looked at the code base in a bit, so my apologies if there's already something in place that does that.

[zador.blood.stained]

Adding per-board patches to ATF, u-boot and kernel is also supported right now, but with kernels there is a big backwards compatibility problem - the kernel package name.

While it's possible to compile per-board kernels, integrating them into the apt repository is impossible without changing the names first, and this means that we have to migrate some (if not all) kernel packages to a new naming, ideally without breaking backwards compatibility (which is a bit complicated)

[TonyMac32]

4 hours ago, zador.blood.stained said:

his means that we have to migrate some (if not all) kernel packages to a new naming

ugh, forgot that part.  Perhaps something with all new boards could be decided if that were deemed reasonable. 

 

OK, but beyond that, does organizing things by SoC/board seem interesting?  It would make documentation a bit easier as well, an example board template could be made readily.  Right now, you have to navigate multiple trees, and realize the U-boot burning script is in a config file in the "sources" directory, etc.  It's not the simplest thing to navigate if a new user is not extremely familiar with it.  Adding the u-boot patch for Tinker S, for instance, took far longer than it should have, I had to rename the folder a couple times (I thought I needed a "default" or something in the name, turned out no).  In a device-based directory tree I could just create a "uboot" folder in the SoC folder without being worried about getting the directory name wrong (whoever introduced the SoC had to worry about that.)

 

Possible top level directory structure:

 

bsp        cache        config        lib        output        userpatch
              

 

Patch and blobs are now under bsp arranged by SoC and (where appropriate in a non-breaking the universe way) board (potato, K2 u-boot blob would be in there, etc).  Config now holds board definitions only, ideally merging the board definition with the sources part of the sources file, then moving all of the script-related parts of it to the new bsp tree. Bootenv/bootscripts/fex/kernel configs are likewise arranged in bsp folder by board. 

 

Would be a lot of work, I think it could be done incrementally (start with the patches, then work through the other parts), I just wanted to mention the idea and see if it seemed logical to me relatively late at night...