rfc [RFC] New Naming Convention for Kernel Source Trees

[lanefu]

See conversation in this commit for background

 

The current default, Next, Dev naming convention for the kernel source trees is confusing, and a bit inconsistent.   I think it's time to rename them and provide clear definitions of what they are supposed to provide.

 

I'll start with some ideas here:

Vendor 

 

This Kernel is currently called Default .  The Vendor kernel would be the BSP-derived kernel or vendor maintained kernel.  Typically there for basic functionality early in a boards life.  Armbian can supply patches for these kernels

 

LTS

This kernel is currently called Next.   The LTS kernel would follow a LTS version of mainline or fork.  Ex: 4.19 of Megous's branch for orangepi, mainline:4.19 for espressobin.   This should be our flagship kernel and the only one we "support" for end users.  Armbian can supply patches for these kernels

 

Unstable

 

This kernel is currently called Dev.  The Unstable kernel would follow a recent branch or fork of a kernel.  Armbian can supply patches for these kernels.

 

Mainline

 

This is new.  It would follow mainline:master.  Its purpose is to provide a 100% mainline experience when practical.  No patches will be supplied by Armbian.

[Tido]

@martinayotte, @TonyMac32, @Igor, @chwe, @gprovost, now that you look at it, do you like the idea or do you see some obstacles ?

 

[martinayotte]

4 hours ago, Tido said:

now that you look at it, do you like the idea or do you see some obstacles ?

For me, it is only "naming convention" ... So, I will continue to work on "unstable" images ...

[gprovost]

I do agree that kernel branch naming is a bit confusing... or rather how the original definition applies in today reality.

 

In mvebu family, there is no more vendor kernel in usage, and I actually I do wonder how many board still use vendor kernel ?

In mvebu family, all kernel branch (default, next and dev) are all actually mainline... and both Default and Next are Longterm.

 

I don't know if the discussion extend to U-Boot ? Because there is also a mix of vendor and mainline. And you have case (more than few) where Kernel is mainline but U-Boot is vendor.

 

I don't really see the point of a dedicated 'mainline' branch, usually the dev branch (what you call unstable) will be the most recent LK mainline version without any patches at first, then slowly maintainer will add patches to make this new mainline usable with this or that boards... unless the board support is already upstream. Once the dev branch is stable enough it should become the next NEXT and so on.

 

Also let's see the reality of most vendor, after 18 months they released their product they are already moving on on their next product, so any vendor repo they have created is usually not maintained. It's why there shouldn't be any concept of vendor branch. What we aim at providing is stable images with as much hardware support as possible... that those images use a vendor kernel or vendor u-boot it's doesn't really matter... For user what actually matters is which LK version and which OS he is running end of the day... and maybe he cares a tiny bit about the U-Boot version. He doesn't really care where it comes from, vendor or not vendor.

 

@lanefu Even though I like your naming/definitions, I think there shouldn't be more than 2 or 3 kernel tree variants. 4 variants will not only make it confusing for user but for developer to.

We need to emphasis user perspective. What a user wants in 99% of the case is an image with the best comprise between hardware support / stability / up-to-date kernel. It's not normal that a user is offered with more than 3-4 different images choice (I include OS variant stretch, buster, bionic in the number of choices).

 

Following your naming / definition suggestion, i would rather see it that way :

 

- LTS ( previously called Default )

- Next

- Dev

 

As said I'm talking from mvebu perspective... where we have not more LK vendor version and if all goes well. So for us losing Default name and current definition make sense. Then introducing LTS naming and define a clear line between what is LTS and what is Next is what interest me.

 

There should be rolling mechanism LTS <- Next <- Dev

In perfect world, each Dev versions becomes Next when providing as much hardware support as current Next. And at some point we elect a Next version to  become the new LTS, and so on.

 

To finish :p Even though I understand the intention to decouple those branch form the OS version... I still think we should a minimum of correlation there. But maybe we keep that for later.

 

 

 

[Tido]

On 7/31/2019 at 1:32 AM, lanefu said:

The current default, Next, Dev naming convention for the kernel source trees is confusing

 

On 7/31/2019 at 1:32 AM, lanefu said:

Vendor

This Kernel is currently called Default .

@gprovost, reading your post I got the impression you stumbled here.

 

Asus Tinkerboard, Rock Pi, Le Potato, all started with VENDOR image and a few tweaks - this justifies for me the right to exist, even a necessity.

The Debian project uses 'unstable' (aka Sid) and so 'everybody' is familiar with this terminology of "it works, but can break at any update" ;  whereas NEXT is a bit vague to interpret.

 

For example, in our company we speak about these three - it doesn't need an explanation, this is just understandable:  Development | Acceptance | Production

To keep it understandable and because we are rolling and do not really have LTS (proof me wrong ) I suggest STABLE instead of LTS.

EDIT:   STABLE based on LTS Kernel. 

Naa, it is getting difficult. I would call it stable and in the documentation explain these are based on LTS Kernel.

 

Edited August 4, 2019 by Tido
see above

[Igor]

sun8i, sun7i pine64 ... Is it legacy? Yes. Is it Vendor? Yes/No. Is it perhaps LTS? All three? Technically is severely fixed vendors kernel by us and sunxi community ... which left that kernel many years ago. This exception is most bizarre, but its real, soon not needed? But when? @chwe already wanted to do that and we can slowly make a plan and do it. This simplified this topic problem.

Mainline. Technically no kernel is pure mainline. Changes goes from little (kernel config, added wireless driver) to severe with 100+ patches. Can this be called mainline? Yes and no. 

 

Adding u-boot or lower firmware into this picture is IMO adding unnecessary complexity. In most cases its kernel independent unless not. So it should remain attached to the kernel naming even its always the same. Whatever that is.

 

In the future, there will be one generic arm32 and one arm64 kernel ... but the need to divide them on maturity/functionality will remain.

 

Rule of three is the key point and naming can actually be pretty generic. https://en.wikipedia.org/wiki/Rule_of_three_(writing)

My 5 cents.

[ning]

this topic is started by my question on github.

 

I have different opinion on naming, policy. here is my point:

 

1, default ---> verdor provided kernel. keep maintain it with additional patches until mainline/LTS kernel fully supported. if no verdor kernel then just drop.

 

2, LTS ---> latest LTS kernel with patches to support full functions (if possible). maintainers to decide whether to switch to next new LTS. Armbian maintians all patches, until upstream LTS support full functions.

 

3, mainline ---> always latest stable kernel branch with patches support basic function, which depends on maintainers efforts.

 

however, kernel patches can be maintained within build script git or seperate kernel fork, depends on whether Armbian has/wants the control of the git repo.

 

while currently, legacy sunx8i kernel is a git repo under Armbian, I don't know why don't merge patches for sun8i legacy kernel to this git?

and sunxi next/dev kernel is a 3rd party kernel fork with addition patches, while next branch is not maintained by that fork owner, armbian should switch to upstream LTS branch instead of to submit LTS update patch to build script, or to fork that 3rd party kernel fork and maintain it under your org.

 

For better maintain patches, below are my suggestions:

 

for vendor/default kernel, because it's old and maybe EOL, its better to have a seperate branch, to enable new board/feature/security bugs.

if vendor still maintain it, keep patches in build scipt.

 

for LTS kernel because it's internal API is stable during its life time, it's better to have a seperate kernel fork to track all patches, and all platfroms which use same LTS version can share same kernel repo and branch, thus may help to save maintaince time. eg, many platform have some common patches: dts overlay, armbian logo, they exist multiple times, if platfrom LTS kernels share same repo/branch, then these common patches no need to be added multiple times. once you have done enabling, the only thing you need to do is merge LTS update to this repo. For rebase to next LTS, the effort is same as maintain patches in build script repo.

 

for mainline kernel, because it's rolling and internal API is not stable, thus can't maintain a seperate branch, it's better to maintain patches in build script git repo.

 

if we can do this, only mainline kernel patch still need to be in build script. thus make build script be pure build script.

[lanefu]

Currently I feel like NEXT is the most confusing type out of all--Especially with Sunxi.... As far as default.. we ought to default to a stable mainline

 

I really like @Tido thoughts regarding the debian-style convention:

 

Legacy - Vendor / Legacy / BSP abomination 

Stable - A kernel that at least follows a recent lts mainline version, even if fork, and patched accordingly (this should be the one we should use as our default image build)

Unstable - Dev Kernel.

[gprovost]

I guess we will all have different opinions on the matter but I feel we still forgot to see it from user perspective.

 

I don't really agree on using debian-like denomination for out use case. We are talking kernel here with a naming system to measure hardware support maturity. It's not because a kernel only supports 60% of the feature of a board that it means it's Unstable. Same the other way it's not because a kernel version support 80% of the feature that it means it's Stable. Using Debian convention will just confuse user because the meaning in reality will be pretty different.

 

I agree that Next is the most confusing name as for now.

 

As for Default, I like the idea of renaming it Legacy, and loosen up a bit the definition... because in my case Helios4 has nothing to do with Vendor BSP.

 

Again let's take a one concrete example. Helios4. latest release (v5.91) is as follow

• Default : U-Boot 2018.11 + LK Mainline 4.14.y (100% hardware support)
• Next : U-Boot Mainline 2019.04 + LK Mainline 4.19.y (100% hardware support)
• Dev : U-Boot Mainline 2019.04 + LK Mainline 5.x

Let's put aside Dev, because personally I feel Dev meaning is pretty clear... and end of the day we don't publish dev image. Again let's look at it from user perspective, we aren't building image just for ourselves.

If today I publish 2 images : Helios4 Default Buster and Helios4 Next Buster, what would drive a User to choose Default over Next ?

I mean both have 100% hardware support, both are based on LK mainline and LK version that longterm ( actually LK 4.14 is much more Longterm than 4.19 if you look at the LK calendar) and both use latest Debian. You might say the choice is pretty trivial, user should pick up Next. But some users might ask themselves shouldn't I use Default since it seems to be the default choice (or in other word recommanded choice).

 

Maybe it won't sound fancy, but I feel it's the following naming that matches the closest to what the builds really are, and it should be quite clear to user what is what and what he should pick. (Ok maybe it applies better to Helios4 use case than other).

 

- Legacy

- Current

- Dev

 

[lanefu]

22 hours ago, gprovost said:

don't really agree on using debian-like denomination for out use case. We are talking kernel here with a naming system to measure hardware support maturity. It's not because a kernel only supports 60% of the feature of a board that it means it's Unstable. Same the other way it's not because a kernel version support 80% of the feature that it means it's Stable. Using Debian convention will just confuse user because the meaning in reality will be pretty different.

 

Fair point.

 

22 hours ago, gprovost said:

Maybe it won't sound fancy, but I feel it's the following naming that matches the closest to what the builds really are, and it should be quite clear to user what is what and what he should pick. (Ok maybe it applies better to Helios4 use case than other).

 

- Legacy

- Current

- Dev

 

I like. I can certainly agree to that.

 

On 8/4/2019 at 11:30 AM, ning said:

3, mainline ---> always latest stable kernel branch with patches support basic function, which depends on maintainers efforts.

 

I see where you're coming from, and I do feel like that's a 4th category in the context of certain boards... (sunxi) as somewhere in between next and dev... and sometimes ahead.      For dev we only use out of tree branches when there's additioanl forward ports being applied by a maintainer.. as mentioned that's mostly sunxi.  for most other boards, its generally a mainline checkout.

 

 

[ning]

@lanefu you can't make everyone happy, keep current is not a bad chioce. if you/Armbian team wants a change, why not something big, fundamental change.

[lanefu]

1 hour ago, ning said:

@lanefu you can't make everyone happy, keep current is not a bad chioce. if you/Armbian team wants a change, why not something big, fundamental change.

 

I like the compromise, I think it's lead to a better solution.     There's a time and a place for big changes, I don't think this is one of them.     We really need to build up more strength as a team before taking on any big changes.   We need small wins and improvements to build that strength.

 

[Igor]

On 8/5/2019 at 4:51 AM, gprovost said:

- Legacy

- Current

- Dev

Best so far I would say.

 

1 hour ago, ning said:

why not something big, fundamental change.

Even we are close or at consensus, let's keep it open. There is no rush to implement this. I would prefer to clean up some other stuff before jumping on this.

[lanefu]

1 hour ago, Igor said:

Even we are close or at consensus, let's keep it open. There is no rush to implement this. I would prefer to clean up some other stuff before jumping on this.

 

thats cool.. are we far enough along in the conversation that I should at least put a note on our project board?

[Igor]

13 minutes ago, lanefu said:

are we far enough along in the conversation

From my perspective yes. Technical implementation discussion can also be done later.

[lanefu]

created card https://github.com/orgs/armbian/projects/1#card-24886141 will convert to issue when we're more ready

[Igor]

1/2 into that direction.

 

[lanefu]

1/2 into that direction.
 

Looks great@igor!

[Igor]

What about proceeding from vendor named kernels toward generic arm64 / armhf ? I think we should start doing this with 5.3.y ... sunxi64+meson64+rockchip64 -> "armbian64" This is something we have to start doing at some point and can be a first step toward this goal.

[Tido]

6 hours ago, Igor said:

vendor named kernels toward generic arm64 / armhf ?

I maybe misunderstand you, but looking at the first posting it was not about that. It was one level higher, you are already at lower, to my understanding.

But before we start here, we should define the toplevel.

 

[Igor]

1 hour ago, Tido said:

but looking at the first posting it was not about that.

 

What I propose is to introduce new generic kernel family which can be implemented right away. It does not need any backward compatibility and lots of testings which is needed for all others.

 

Starting with current stable kernel (5.3.y).

[ning]

glad to see brand new start. how to maintain it? patches or new kernel tree? 

[Igor]

2 hours ago, ning said:

patches or new kernel tree

 

I vote to proceed with patches but before getting to that questions, we have to cleanup:

https://github.com/armbian/build/tree/master/config/sources

 

I was staring into this for a while and still don't find the best way to prepare for new kernel naming and generic kernels. Possible first step would be to clean out legacy/deprecated stuff. And rethink the whole config organisation: boards - board families (+ kernel families) - architectures ...

 

 

[Igor]

Perhaps bringing arm64 this way in? (this should be "current")

https://github.com/armbian/build/compare/arm64

Regarding naming adjustments I haven't figure out best way.

[Tido]

1 hour ago, Igor said:

arm64 this

could you create a table in this post and show the structure you are looking for.  PS: make the table tooo large, you cannot make it bigger in the process.

 

[chwe]

On 8/6/2019 at 3:55 PM, Igor said:

On 8/5/2019 at 4:51 AM, gprovost said:

- Legacy

- Current

- Dev

Best so far I would say.

- Legacy

- Stable

- Dev

 

Or we have a 'policy' change (I would be slightly in favor). Keep the current 'next' branch on a LTS kernel and move only with dev. Currently all our 'favored' platforms are more or less mature in mainline (RK3399, Sunxi, and as far as I can understand meson - btw if we go for it, I'll add the missing bits for mt7623 as well currently it doesn't need many patches). So I see a chance here to bring those together (it's still a bunch of work but it might work - for sure every patch which touches non SoC specific files must be checked). I'm quite sure this isn't a small goal (bringing all kernelconfigs together and test if they're not incompatible to each other), so we might start on 5.3 with the goal to get it mature with the next LTS kernel (5.4 should be the next LTS).

 

On 9/29/2019 at 7:52 PM, Igor said:

Possible first step would be to clean out legacy/deprecated stuff. And rethink the whole config organisation: boards - board families (+ kernel families) - architectures ...

this seems to be a great goal.

[Igor]

27 minutes ago, chwe said:

so we might start on 5.3 with the goal to get it mature with the next LTS kernel

Exactly. 

If we leave current naming as is and move toward with armhf/arm64 we can do whatever we like. 1st generic build, sunxi64-dev + meson64-dev =

dpkg-deb: building package 'linux-headers-arm64-stable' in '../linux-headers-arm64-stable_5.98_arm64.deb'.
dpkg-deb: building package 'linux-dtb-arm64-stable' in '../linux-dtb-arm64-stable_5.98_arm64.deb'.
dpkg-deb: building package 'linux-image-arm64-stable' in '../linux-image-arm64-stable_5.98_arm64.deb'.
dpkg-buildpackage: info: binary-only upload (no source included)
dpkg-deb: building package 'linux-source-5.3.3-default-arm64-stable' in '/home/igorp/h6/.tmp/linux-source-default-meson64_5.98_all.deb'.

or it should be DEV? IMO no. Perhaps current is better naming since "stable" is hard to say. Kernel might be stable for most but Allwinner H6 (for example). This job is quick, quicker if we just "rename" sunxi64 -> arm64 and leave others as is and bring them in slowly.

[chwe]

4 minutes ago, Igor said:

better naming since "stable" is hard to say.

vanilla... (and all bsp kernels are called chocolate )

 

stable might not be a good term here.

 

7 minutes ago, Igor said:

This job is quick, quicker if we just "rename" sunxi64 -> arm64 and leave others as is and bring them in slowly.

ask the 'sunxi-gang' e.g. @martinayotte and @jernej.

 

I think the 'glue it together' part is done quick.. Testing might be the harder part here.

[martinayotte]

25 minutes ago, chwe said:

ask the 'sunxi-gang'

I don't know what to answer ...

For me, it doesn't matter much since my works are usually always in DEV.

(BTW, Megous has switched to 5.4-rc1 ...  )

[Igor]

18 minutes ago, chwe said:

Testing might be the harder part here.

Testing is always hard. Especially without extensive community support or by non-existing QA team running premade autotests.

 

People takes https://www.kernel.org/ "stable" as stable  which only extends the confusion. We can't say legacy, nor vanilla, nor cocos, nor stable ...

 

1 minute ago, martinayotte said:

it doesn't matter much since my works are usually always in DEV.

The question is shell we now/today move sunxi64-dev with 5.3.y under arm64-current common branch ... which is IMO better than 4.19 on most of Allwinner arm64 hw.