rpardini

Hello folks, sorry for the absence, I've been facing connection issues. Let me try and explain a bit, from the beginning. I will focus on kernel, but most applies also to u-boot, firmware, etc; also keep in mind that not all packages have been migrated to the new scheme: BSPs, Desktop BSPs, etc, are not yet changed (too many things to hash / little benefit / too many things to decide before that). The reason we've a new version scheme is for consistent caching and fast CI runs. The important part about the caching is that we need to obtain the "desired" version of the kernel, **before** actually git clone/fetching the sources, before patching, and before compiling; otherwise it's useless: if the user/developer has to do any/all of this to get to the version, then it's "too late" for caching. This needs to be crystal clear, otherwise everything just looks like an insane person did random stuff and broke Versioning for no reason. Important: the caching part benefits some users/builders, in the sense that someone casually building an image at home might hit the cache, but the main target/reason it exists is for our (Armbian) own Continuous Integration (CI): every PR, every nightly, every release, etc, should benefit from the caching, but it has to be consistent (not use wrong version), and hit ratios should be high. Before this scheme, we had only REPOSITORY_INSTALL mechanism, which frequently installed wrong/mismatched versions, we just said "use whatever is in the apt repo", which could be literally anything, and frequently led to half-working images. Also, "apt repo" is a very inflexible/demanding solution in terms of infrastructure, which requires HTTPS servers/mirrors, Release lists, etc, and is not available for free for every developer/partner. New caching uses standard OCI registries, which are available for anyone to host, and even hosted for free on GitHub (ghcr.io) and DockerHub among others. That said, let's get to how we calculate the version for the kernel package. The code, containing a long explanation, is at https://github.com/armbian/build/blob/main/lib/functions/artifacts/artifact-kernel.sh - but in essence goes like this: 1) "base_version" (bv): Get the KERNELSOURCE and KERNELBRANCH variables (usually set in family config file for the board being built). Use `git ls-remote` to obtain the SHA1 of the commit of that. Obtain the HTTP URL for the Makefile at that commit. ("View raw" in GitHub web interface, but also for GitLab, and others). Get the Makefile contents (using HTTP, not Git, since there is not "partial git fetch"). Parse the Makefile for the version information inside, including the "codename". (Codename here is just to "prove" that data comes from Makefile, not "git tag"). Cache all this on disk for a few minutes. 2) "S"ha1: just the SHA1 of the commit mentioned above. 3) "D"rivers: Obtain a SHA256 hash of the "drivers". Kernel drivers (EXTRAWIFI etc) are just a complex variation of patches; they are shared across families, and are not stored as patches, but in the end are transformed into one huge patch, so behave like one. 4) "P"atches: Obtain a SHA256 hash of the "patches": simple hashing of the patches/series/etc under KERNELPATCHDIR (and userpatches variations). If no patches, "0000" is used. 5a) "C"onfig: Obtain a SHA256 hash of the ".config" file that is gonna be used for the build. LINUXCONFIG (and userpatches variations). 5b) "H"ooks: Obtain a SHA256 hash of the hooks that might modify the .config. (This is a bit more complex to explain, since it involves extension mechanism, which is out of scope here). Suffice to say that if you change a hook that changes kernel configuration, the version should change. 6) "B"ash: Some lib/framework code, if changed, should cause the version to change as well. Not all bash code, but bash that calls the "make", or does the packaging, if changed, need to change the version. Currently "lib/compilation/kernel*.sh" is hashed. Hopefully, you can now understand that `Version: ` below. Each SHA256 hash is shortened to a mere 4 characters: So now we've to address two things: a) the original question by @belegdol; b) the general question of "how to control upgrades via apt". ------------------------------------------------------------------------ Ref the "odroidxu4" situation: @belegdol maintains that kernel family, in the following way (please correct me if I'm wrong): a) Vendor, HK, has an Git repo, which has a 5.4.y version, plus HK/Exynos patches. That Makefile currently says "5.4.228". It has heavily lagged behind official releases in the past AFAIK but seems pretty recent recently. HK "merges" updates from mainline, instead of cleanly rebasing, though, since they've lost all hope of mainlining their stuff and merging is easier. b) Belegdol extracts new/recent 5.4.y patches from upstream/kernel.org, (possibly?) manually fixes them so they apply cleanly on top of HK kernel, adds them to Armbian via PRs. Those patches modify the Makefile so the version is incremented. So AFAIK this is the only kernel family that is done this way (we did some similar stuff for rk3588 legacy, but thankfully no more). Every other family works in reverse: we start with a pristine/clean/updated kernel.org branch (whose Makefile is updated), and then patch what is needed on top of that. That way the kernels "auto update", unless some patch breaks. Hopefully it is clear, together with the above explanation, why the "wrong" version is used: only the original Git's Makefile version (not tag... but tag's Makefile's contents) is considered, since patches are only ever applied after Version is calculated, and cache possibly hit or missed. So while I confess I had not xu4 as the primary target of this whole thing, it representing less than 1% of all boards. Still, there needs to be a way to make it work, we just need to decide how. One possibility: invert the XU4 patching scheme, by rebasing HK's branch onto an upstream tag, and extracting the patches (git format-patch); then change our KERNELSOURCE/KERNELBRANCH to mainline, and let it auto update. This might, or not, be a lot of work initially, and might, or not, impose a lot less work over time. It depends on how many changes there are in HK Git, if the relevant patches are constantly updated, etc. From what I hear HK has little work done on the Exynos, but I am not an expert. I would like this scheme since it would align XU4 with the rest. Another possibility: introduce a way to (optionally) override the "base_version" (example, variable FORCE_KERNEL_EXACT_VERSION_VIA_PATCHES="5.4.232") directly in the family configuration, use that for the artifact version if present, and change nothing else. This is _more_ work for xu4 maintainer, and might be get out-of-sync with reality (change patches, but not change variable, and vice versa). This might be useful also for other situations, although none come to mind right now, it might be useful for Igor's "emergencies" 😉 ------------------------------------------------------------------------ Now about "how to control upgrades via apt". To understand this, we need to understand the formally defined Debian Policy about the "Version: " field in the DEBIAN/control file. I recommend we all read https://www.debian.org/doc/debian-policy/ch-controlfields.html#s-f-version -- it is one of the more dense, complex, and important paragraphs of text ever written by Debian, and defines how apt decides what version is "higher" than another. There, we can see there are 3 components, each with its own rules, and the special rule about the "~" (tilde) character. The important part: I did not want to "impose" the final apt versioning scheme before we talked about it. Also: there's pending work in the general area of "repository management/publishing" that needs to be done before we can publish _anything_, consistent or not, to any apt repository. I was told we had until May/2023 (so circa 2 months) to reach consensus and implement those -- but now it seems the sky is falling down much earlier? So current "Version: " is a non-working, non-deterministic, inconsistent situation in regards to apt upgrades: if you try to predict how our "bv+S+D+P+CH+B" will be handled by apt, you will soon realize it's inconsistent: "fixed"/later patches on same base version might get downgraded to broken ones, and vice versa, almost randomly (as is the property of SHA256 hashes). One extremely easy way to "fix it" is to prefix our current "bv+S+D+P+CH+B" with the $REVISION (which comes from VERSION file or equivalent userpatches) and a tilde. That would force apt to respect the "old" versioning scheme, while still keeping caching consistent. Unfortunately, doing that would drastically decrease cache hit ratio, since changing VERSION file would cause an otherwise identical kernel to be ignored/rebuilt. That might happen because user/dev changed userpatches/VERSION, but is otherwise identical to main, or because we branched off a release/tag (otherwise identical to main at that instant). But it might be the best option right now, especially if no-one else is willing to invest brain time on this. I could then twist my own brain on how to recover the cache hit ratio somehow else (eg, search cache for both with-$REVISION and without-$REVISION variants, invent some "IS_RELEASE=1" variable, doing GitHub Actions acrobatics, etc; is a lonely world where only Igor and Ricardo live). Similar option: transforming $REVISION ("23.05.01") into an integer (230501) and using that as the "epoch" part of Version field, would also work, but same cache-non-hitness applies. ----- Phew, that was long. If you read all this, give yourself a pat in the back. Let me know what you think on all the different points... @going @Werner @belegdol @Igor @SteeManand everyone else interested.

That would be helpful, yes, thanks. To reproduce the problem, you can delete `cache/tools/oras` and the bug should appear again...

Rock 5b test images, from armbian-next. Using vendor u-boot + patches, and vendor kernel, both straight from radxa's git XFCE desktop with "browsers" app group - https://github.com/rpardini/armbian-release/releases/download/20220710b/Armbian_20220710b-rpardini_Rock-5b_jammy_legacy_5.10.66_xfce_desktop.img.xz CLI - https://github.com/rpardini/armbian-release/releases/download/20220710b/Armbian_20220710b-rpardini_Rock-5b_jammy_legacy_5.10.66.img.xz PR: https://github.com/armbian/build/pull/3984 (against master) All work by @amazingfate I just gathered stuff and built images. Thanks to @monkaBlyat @lanefu @piter75 @amazingfate for tests / patches etc