rpardini

Hello folks, sorry for the absence, I've been facing connection issues. Let me try and explain a bit, from the beginning. I will focus on kernel, but most applies also to u-boot, firmware, etc; also keep in mind that not all packages have been migrated to the new scheme: BSPs, Desktop BSPs, etc, are not yet changed (too many things to hash / little benefit / too many things to decide before that). The reason we've a new version scheme is for consistent caching and fast CI runs. The important part about the caching is that we need to obtain the "desired" version of the kernel, **before** actually git clone/fetching the sources, before patching, and before compiling; otherwise it's useless: if the user/developer has to do any/all of this to get to the version, then it's "too late" for caching. This needs to be crystal clear, otherwise everything just looks like an insane person did random stuff and broke Versioning for no reason. Important: the caching part benefits some users/builders, in the sense that someone casually building an image at home might hit the cache, but the main target/reason it exists is for our (Armbian) own Continuous Integration (CI): every PR, every nightly, every release, etc, should benefit from the caching, but it has to be consistent (not use wrong version), and hit ratios should be high. Before this scheme, we had only REPOSITORY_INSTALL mechanism, which frequently installed wrong/mismatched versions, we just said "use whatever is in the apt repo", which could be literally anything, and frequently led to half-working images. Also, "apt repo" is a very inflexible/demanding solution in terms of infrastructure, which requires HTTPS servers/mirrors, Release lists, etc, and is not available for free for every developer/partner. New caching uses standard OCI registries, which are available for anyone to host, and even hosted for free on GitHub (ghcr.io) and DockerHub among others. That said, let's get to how we calculate the version for the kernel package. The code, containing a long explanation, is at https://github.com/armbian/build/blob/main/lib/functions/artifacts/artifact-kernel.sh - but in essence goes like this: 1) "base_version" (bv): Get the KERNELSOURCE and KERNELBRANCH variables (usually set in family config file for the board being built). Use `git ls-remote` to obtain the SHA1 of the commit of that. Obtain the HTTP URL for the Makefile at that commit. ("View raw" in GitHub web interface, but also for GitLab, and others). Get the Makefile contents (using HTTP, not Git, since there is not "partial git fetch"). Parse the Makefile for the version information inside, including the "codename". (Codename here is just to "prove" that data comes from Makefile, not "git tag"). Cache all this on disk for a few minutes. 2) "S"ha1: just the SHA1 of the commit mentioned above. 3) "D"rivers: Obtain a SHA256 hash of the "drivers". Kernel drivers (EXTRAWIFI etc) are just a complex variation of patches; they are shared across families, and are not stored as patches, but in the end are transformed into one huge patch, so behave like one. 4) "P"atches: Obtain a SHA256 hash of the "patches": simple hashing of the patches/series/etc under KERNELPATCHDIR (and userpatches variations). If no patches, "0000" is used. 5a) "C"onfig: Obtain a SHA256 hash of the ".config" file that is gonna be used for the build. LINUXCONFIG (and userpatches variations). 5b) "H"ooks: Obtain a SHA256 hash of the hooks that might modify the .config. (This is a bit more complex to explain, since it involves extension mechanism, which is out of scope here). Suffice to say that if you change a hook that changes kernel configuration, the version should change. 6) "B"ash: Some lib/framework code, if changed, should cause the version to change as well. Not all bash code, but bash that calls the "make", or does the packaging, if changed, need to change the version. Currently "lib/compilation/kernel*.sh" is hashed. Hopefully, you can now understand that `Version: ` below. Each SHA256 hash is shortened to a mere 4 characters: So now we've to address two things: a) the original question by @belegdol; b) the general question of "how to control upgrades via apt". ------------------------------------------------------------------------ Ref the "odroidxu4" situation: @belegdol maintains that kernel family, in the following way (please correct me if I'm wrong): a) Vendor, HK, has an Git repo, which has a 5.4.y version, plus HK/Exynos patches. That Makefile currently says "5.4.228". It has heavily lagged behind official releases in the past AFAIK but seems pretty recent recently. HK "merges" updates from mainline, instead of cleanly rebasing, though, since they've lost all hope of mainlining their stuff and merging is easier. b) Belegdol extracts new/recent 5.4.y patches from upstream/kernel.org, (possibly?) manually fixes them so they apply cleanly on top of HK kernel, adds them to Armbian via PRs. Those patches modify the Makefile so the version is incremented. So AFAIK this is the only kernel family that is done this way (we did some similar stuff for rk3588 legacy, but thankfully no more). Every other family works in reverse: we start with a pristine/clean/updated kernel.org branch (whose Makefile is updated), and then patch what is needed on top of that. That way the kernels "auto update", unless some patch breaks. Hopefully it is clear, together with the above explanation, why the "wrong" version is used: only the original Git's Makefile version (not tag... but tag's Makefile's contents) is considered, since patches are only ever applied after Version is calculated, and cache possibly hit or missed. So while I confess I had not xu4 as the primary target of this whole thing, it representing less than 1% of all boards. Still, there needs to be a way to make it work, we just need to decide how. One possibility: invert the XU4 patching scheme, by rebasing HK's branch onto an upstream tag, and extracting the patches (git format-patch); then change our KERNELSOURCE/KERNELBRANCH to mainline, and let it auto update. This might, or not, be a lot of work initially, and might, or not, impose a lot less work over time. It depends on how many changes there are in HK Git, if the relevant patches are constantly updated, etc. From what I hear HK has little work done on the Exynos, but I am not an expert. I would like this scheme since it would align XU4 with the rest. Another possibility: introduce a way to (optionally) override the "base_version" (example, variable FORCE_KERNEL_EXACT_VERSION_VIA_PATCHES="5.4.232") directly in the family configuration, use that for the artifact version if present, and change nothing else. This is _more_ work for xu4 maintainer, and might be get out-of-sync with reality (change patches, but not change variable, and vice versa). This might be useful also for other situations, although none come to mind right now, it might be useful for Igor's "emergencies" 😉 ------------------------------------------------------------------------ Now about "how to control upgrades via apt". To understand this, we need to understand the formally defined Debian Policy about the "Version: " field in the DEBIAN/control file. I recommend we all read https://www.debian.org/doc/debian-policy/ch-controlfields.html#s-f-version -- it is one of the more dense, complex, and important paragraphs of text ever written by Debian, and defines how apt decides what version is "higher" than another. There, we can see there are 3 components, each with its own rules, and the special rule about the "~" (tilde) character. The important part: I did not want to "impose" the final apt versioning scheme before we talked about it. Also: there's pending work in the general area of "repository management/publishing" that needs to be done before we can publish _anything_, consistent or not, to any apt repository. I was told we had until May/2023 (so circa 2 months) to reach consensus and implement those -- but now it seems the sky is falling down much earlier? So current "Version: " is a non-working, non-deterministic, inconsistent situation in regards to apt upgrades: if you try to predict how our "bv+S+D+P+CH+B" will be handled by apt, you will soon realize it's inconsistent: "fixed"/later patches on same base version might get downgraded to broken ones, and vice versa, almost randomly (as is the property of SHA256 hashes). One extremely easy way to "fix it" is to prefix our current "bv+S+D+P+CH+B" with the $REVISION (which comes from VERSION file or equivalent userpatches) and a tilde. That would force apt to respect the "old" versioning scheme, while still keeping caching consistent. Unfortunately, doing that would drastically decrease cache hit ratio, since changing VERSION file would cause an otherwise identical kernel to be ignored/rebuilt. That might happen because user/dev changed userpatches/VERSION, but is otherwise identical to main, or because we branched off a release/tag (otherwise identical to main at that instant). But it might be the best option right now, especially if no-one else is willing to invest brain time on this. I could then twist my own brain on how to recover the cache hit ratio somehow else (eg, search cache for both with-$REVISION and without-$REVISION variants, invent some "IS_RELEASE=1" variable, doing GitHub Actions acrobatics, etc; is a lonely world where only Igor and Ricardo live). Similar option: transforming $REVISION ("23.05.01") into an integer (230501) and using that as the "epoch" part of Version field, would also work, but same cache-non-hitness applies. ----- Phew, that was long. If you read all this, give yourself a pat in the back. Let me know what you think on all the different points... @going @Werner @belegdol @Igor @SteeManand everyone else interested.

That would be helpful, yes, thanks. To reproduce the problem, you can delete `cache/tools/oras` and the bug should appear again...

Might be minimal image and firstlogin don't like each other? Either way, please open GitHub issue with those, can't keep up with 2 places for issues.

This has been handled for other cases where `git format-patch` was used (kernel drivers cache, I think) so should also be fixed: it should use MAINTAINER and MAINTAINEREMAIL (default to Igor) if not set on git config. git config might not exist at all for example if using Docker -- so this is smth that should be handled anyway. Can you open a GitHub issue? I am losing track of stuff by splitting between forum and GH....

Even after ruminating through the codebase for years, this is new to me. Will investigate..

Thanks @schwar3kat ! This has been elusive, but its clear that when the tooling is 1st downloaded, the parsing fails. 2nd run the tooling is already there so doesnt fail. Will try to address!

This week's shortlog. Really short, this time, since most of the artifacts stuff isn't included yet. Ricardo Pardini <ricardo@pardini.net> (19): armbian-next: tmpfs: use `size=99%` for all tmpfs's; (rootfs/WORKDIR/chroot); thanks Igor for suggestion armbian-next: armbian-kernel: default .config options set for all kernels armbian-next: git: backport `safe.directory` method from master (just add `*`, once) -- thanks Igor armbian-next: logging: introduce `SHARE_LOG=yes` to auto-curl the logs to paste armbian-next: logging: introduce `RAW_LOG=yes` to export .tar with raw unprocessed logs (for development) armbian-next: tooling: include ORAS tools in Dockerfile; use them if available instead of downloading armbian-next: tooling: introduce helper/downloader for shellcheck; include in Dockerfile armbian-next: rootfs: include hash of rootfs bash code in the `packages_hash`/`rootfs_cache_id` armbian-next: debs: introduce `shellcheck` for `DEBIAN/postinst` and others, before dpkg build armbian-next: python/pip: include pip pkgs in Dockerfile; use them if available instead of downloading armbian-next: rootfs: fix: too many bash files included in rootfs `B` hash, reduce to only what is used to produce rootfs tarball armbian-next: qemu-uboot-xxx: bump to 23.01; use GitHub source for u-boot armbian-next: binfmts: suppress error message when trying to emulate armhf on arm64 armbian-next: git/ref2info: fixes for using `commit:<sha1>` armbian-next: kernel: pkg: dtb: show 2 levels of dtbs in tree armbian-next: bsp-cli: fix shellcheck warning armbian-next: armbian-kernel: force `CONFIG_LOCALVERSION_AUTO=n` and `DEBUG_INFO=n` Docs: slides (consolidated, 08/Feb/2023) ------------------------ armbian-next END marker ----------------------------- Kreyren <kreyren@fsfe.org> (3): boards/README: Document SKIP_ARMBIAN_REPO `bookworm`: has new/separate repos for non-free-firmware (#4801) [cherry-pick] distro-agnostic: `dropbear-initramfs`: make sure directory exists SteeManMI <barrylind@yahoo.com> (1): `meson64`: `current`: Enable MGLRU on meson64-current [cherry-pick] amazingfate <liujianfeng1994@gmail.com> (1): `rockchip64`: update bl31 and ddr blobs of rk3568 [cherry-pick]

Yep, I worked on that a few months ago. We've something like `./compile.sh distccd` which starts a distcc daemon, complete with zeroconf etc, using the same toolchains Armbian would. Something else (forget what) enables it to be used on a controller node. You do need a bigger controller machine, say 8-cores, to completely max out some two/three 4-core distccd's, due to the high preprocessing load. The network between the machines has to be very good, too (gigabit+). All in all is a "fun" way to use all those SBCs laying around doing nothing, but don't expect something miraculous... One day I'll finish / document this...

Yep, not only are changes to hooks ignored, but changes to core code are _also_ ignored. For now the only way to get around it is to remove the .deb in question from output/debs (or CLEAN_LEVEL=debs, although that cleans too much). Meanwhile I'm working on almost exactly the same subject but for packages like kernel and u-boot, and bsps will come after that. Essentially I'll be hashing the bash code of drivers/patches/config + core+hooks and including that into the .deb Version: field.

Yep, good idea. Commit log is at 400+ commits... I've slides (in Markdown/HedgeDoc) on my own instance, that I've copied to Lane's at https://docs.lane-fu.com/dAGOaCTLQ0SgPnQsHz6LWw?both -- it's in slide style, first few ones are outdated, but can serve as base: https://docs.lane-fu.com/dAGOaCTLQ0SgPnQsHz6LWw?both -- also, if you have patience to watch the developer meeting videos, most secrets will be revealed. I was hoping to do one last consolidated/updated presentation before the switch, but time is pressing...

here's the git shortlog; as mentioned in the meeting, I've reworded quite a bit, so this a lot longer (goes back in time further) than for other meetings. Ricardo Pardini <ricardo@pardini.net> (441): armbian-next: `lib/tools` - python - config definitions capture and massaging armbian-next: `general-packaging`, `mkdebian` and `builddeb` are NO MORE armbian-next: update defaults and comments in config file armbian-next: atf: downgrade gcc errors to warnings, so ATF 2.5, 2.2 and others can be built on modern toolchains armbian-next: fix for .bin/.elf copy from atf to uboot dir armbian-next: kernel: config: `make olddefconfig` (not `oldconfig`, which is interactive) when `KERNEL_CONFIGURE=yes` armbian-next: kernel build: always add container (series/patch-dir) mtime as minimum mtime for patch results armbian-next: do not exclude 'CC' messages from kernel make armbian-next: remove warm/cold bundles from `fetch_from_repo`, replace with 2 invocation-specific hooks and `gitballs` (.tar of .git) armbian-next: try to parse `Source: linux-5.x.y-rcZ` correctly into `VER=5.x.y-rcZ` instead of `VER=5.x.y` armbian-next: allow overriding rkbins repo URL with `RKBIN_GIT_URL` armbian-next: u-boot: introduce `UBOOT_DEBUGGING=yes` which enables `CONFIG_LOG=y` and others armbian-next: atf: add `LOG_LEVEL=40` (up from default 20) and `BUILD_STRING="armbian"` to atf make invocation armbian-next: back to ANSI logs by default; HTML logs only by `EXPORT_HTML_LOG=yes` armbian-next: u-boot: extract into functions; add hooks - for more flexibility armbian-next: python: info: replace instances of ${SRC} with literal `${SRC}` so infos are more comparable armbian-next: `check_loop_device()` now uses `do_with_retries 5 ...` to give the OS time armbian-next: introduce `KERNEL_EXTRA_DIR` to suffix `LINUXSOURCEDIR` forcefully armbian-next: kernel-localmodconfig: update `kernel_config_mtime` after running `localmodconfig`, otherwise changes not detected armbian-next: traps: cleanups: run cleanups in the reverse order they were added; allow running one single cleanup and removing from list armbian-next: cleaning: fix shortcircuits, warn about unknown clean levels armbian-next: fix logging headers in HTML/ANSI log output armbian-next: WiP: u-boot: add fasthash to u-boot armbian-next: u-boot: add debug info when hacking `CONFIG_ENV_IS_IN_EXT4` in u-boot `.config` armbian-next: `write_image_to_device`: if `SKIP_VERIFY=no`, don't waste time checksumming the image, it won't be used armbian-next: logs: avoid errors by cwd'ing to SRC armbian-next: uboot: use CFLAGS/KCFLAGS everywhere; run make through `unbuffer` (from `expect` package) for full color logging armbian-next: kernel: run kernel's make through `unbuffer` as well armbian-next: cleaning: don't complain about 'none' `CLEAN_LEVEL` armbian-next: sources/families: meson64: gx: use `python2` explicitly for calling `acs_tool.pyc`; better logging armbian-next: prepare a bin dir inside WORKDIR with a `python` symlink to `/usr/bin/python2` and add it to `PATH` armbian-next: retry apt update & apt install family packages in addition to board packages armbian-next: fix for tests of `BOOTSCRIPT` and `BOOTCONFIG!=none` armbian-next: split `early_prepare_host_dependencies()` (run early, during config, before aggregation) from `prepare_host()` (run after aggregation) armbian-next: re-add dependency `fdisk` after juggling rebases armbian-next: introduce `ARMBIAN_LOGS_TO_JOURNAL` and `ARMBIAN_LOGS_JOURNAL_IDENTIFIER` (logs `display_alert` to journald) armbian-next: debugs about armbianEnv.txt overlays/fdtfile armbian-next: deploy bootscript even if BOOTCONFIG=none armbian-next: HACK: ROOTFSCACHE_VERSION is undefined and failing everytime armbian-next: fix "create list of installed packages for debug" armbian-next: many fixes after v29 rebase; mostly involving umount armbian-next: let all unmounting of `${MOUNT}` be done by `umount_chroot_recursive` armbian-next: less verbose umount_chroot(), except after first try, so we can know what is left mounted armbian-next: bye bye `$FAST_CREATE_IMAGE`: always use `truncate` for blank image armbian-next: normalize mount/umount `$target`s to have (or not) the trailing slash armbian-next: move `resolv.conf` symlink to systemd to post_debootstrap_tweaks so customize phase has working DNS armbian-next: grub: call `VER="generic" update_initramfs` when `"${DISTRO_GENERIC_KERNEL}" == "yes"` (so ddk builds work again) armbian-next: `umount_chroot_recursive`: don't try to `realpath` unless it's a dir to begin with armbian-next: don't break if parsing `BOOT_SOC` out of `BOOTCONFIG` fails; sanity checks for `BOOT_SOC`/`DDR_BLOB` armbian-next: export `CHOSEN_KERNEL_WITH_ARCH` again, for reporting only armbian-next: `kernel`: add hooks `fetch_sources_for_kernel_driver` and `patch_kernel_for_driver` to allow migrating EXTRAWIFI stuff to extensions armbian-next: logging: don't bomb due to lack of `git` or `zstdmt` armbian-next: traps: don't duplicate error message/stacktrace when error occurs in subshell; instead indicate clearly `SUBSHELL` armbian-next: show `mkfs` ext2/ext4 output (remove `-q`), it's being run under the logging manager already armbian-next: the great cli entrypoint (+docker) rewrite; introduce `USE_LOCAL_APT_DEB_CACHE` replacing `apt-cacher-ng` armbian-next: always include `libc6-amd64-cross` in hostdeps; `qemu` is a dummy package now, remove armbian-next: add `gcc-riscv64-linux-gnu` to hostdeps (`crossbuild-essential-riscv64` is Ubuntu-only) armbian-next: WiP/TODO: `extras/buildpkg.sh` code (eg: chroot_installpackages_local) has not been handled yet, warn and don't run it armbian-next: retry 3 times for loop device's size, then give up; introduce `RETRY_RUNS` in `do_with_retries()` armbian-next: back to including 'cli' `sources/apt` sources when building non-desktops (eg, nala etc) armbian-next: make sure we're running on Bash 5.x at least armbian-next: make sure we've coreutils armbian-next: don't force SHOW_LOG=yes during Dockerfile build; we're verbose enough without it armbian-next: docker: try support Rancher Desktop (in dockerd mode), experimental. armbian-next: docker: don't even try to to prepare Docker, if it's not available armbian-next: cleanup `extensions` logging, using `display_alert` new levels `extensions` (`SHOW_EXTENSIONS=yes`) and `extensionstrace` (`SHOW_EXTENSIONS_TRACE=yes`) armbian-next: cleanup logging in distro-agnostic.sh; don't use `install.log` or such armbian-next: `mountpoints.sh` with all bind/volume directories definitions and looping func; use it to determine best bind/volume combination for each host OS in Docker armbian-next: initial post-`kinetic` `resolved` debacle, for now just WARN don't break; missing resolved package in kinetic? armbian-next: `docker`: handle the case where we can't pull, and don't otherwise have, the base image; in this case use original debian/ubuntu image armbian-next: don't `uuidgen` twice for no reason armbian-next: `docker`: correctly show all Armbian Docker volumes; use bind, not volume, for `output/debs` armbian-next: correctly test for systemd `is-system-running --quiet` armbian-next: better handling for `binfmt_misc` woes; don't require it when native build & only complain if it really fails armbian-next: avoid git cramps when deciding to init repos armbian-next: show more complete gcc versions in logs armbian-next: `cli`: de-hardcode `build` and `docker`'s interdependence, via `DOCKER_CLI_CMD` armbian-next: `cli`: a bit more untangling of re-launching; introduce `ARMBIAN_CLI_RELAUNCH_COMMAND` and `cli_standard_relaunch_docker_or_sudo()` armbian-next: remove usage of `REQUIREMENTS_DEFS_ONLY`; initialize extensions for `requirements` cli command & docker pre-game deps armbian-next: rename `compile_plymouth-theme-armbian()` to `compile_plymouth_theme_armbian()` for consistency armbian-next: `json-info` command, wrapping up the Python tooling; running in Docker; fix config dump armbian-next: introduce hook `post_family_config_branch_<branch>()` as convenience after `post_family_config()` hook armbian-next: docker: locale-gen + enable all existing extensions before calculating docker hostdeps armbian-next: docker: disable showing Docker volume usage; fix syntax in test for `ARMBIAN_RUNNING_IN_CONTAINER` armbian-next: the great aggregation rewrite: pt 0: `SHOW_AGGREGATION` debug logging armbian-next: introduce `ARMBIAN_FIRMWARE_GIT_SOURCE` and `_BRANCH` to allow custom firmware repository coordinates armbian-next: git: do not delete working copies on remote origin mismatch; accept empty origins as valid armbian-next: u-boot: use `scripts/config` instead of `sed`ing into the `.config` (BOOTDELAY, CONFIG_LOGLEVEL=6, etc); v2022.10+ only armbian-next: `distcc`: optionally using `distcc` for kernel build armbian-next: `cli`: `distccd` launcher for build farm armbian-next: better `ccache` logs/stats/Docker-support/use in u-boot/kernel armbian-next: `call trace` analyzer and call-graph generator armbian-next: include `packages.external` and apt `sources` only in DESKTOP builds (pre-`aggregation.py`) armbian-next: `aggregation.py` rewrite; introduce `package-lists.sh`; usage in core lib armbian-next: `aggregation.py` and `package-lists.sh` usage in core grub-* extensions armbian-next: `grub`: fix/hack for grub deploy inside Docker/chroot when /dev/disk/by-uuid is not present; add sanity checks armbian-next: `vmware-vm` core extension: enables `image-output-ovf` and adds some cli and desktop packages to image list armbian-next: `image-output-utm` core extension: produces qcow2-based UTM ZIP image armbian-next: introduce `PREFER_DOCKER`, which acts the same as not-being-root: uses docker always armbian-next: `git`: add `origin` remote if it is empty armbian-next: REVERT: remove the BUILD_ONLY `build-tasks.sh` which makes no sense in armbian-next armbian-next: `qcow2`: introduce `QCOW2_KEEP_IMG=no`, which deletes the original .img after qcow2'ing it armbian-next: hostdeps: python2/3: add `setuptools` (required by some u-boot configs) armbian-next: `u-boot`: introduce hook `pre_config_uboot_target()` armbian-next: `qemu-uboot-arm64` and `qemu-uboot-x86` new boards armbian-next: fix for grub's `DISTRO_GENERIC_KERNEL` on Debian (not Ubuntu) during update-initramfs armbian-next: docker: implement `docker-purge` and `docker-shell` cli commands; deprecated mountpoints armbian-next: logging: helpers for handling special logging if stdout is a terminal (user-interactive) armbian-next: non-Docker cleaning of deprecated mountpoints (before general_cleaning) armbian-next: kernel: use `git worktree` & kernel.org clone bundles for kernel working copies armbian-next: u-boot: use `git worktree` for u-boot; full mainline clone for bare armbian-next: logging: add Markdown logging output armbian-next: git: `fetch_from_repo()`: determine, use for checkout, and set the SHA1 rev in `checked_out_revision` var armbian-next: introduce `INSTALL_HOST_TOOLS=no` to skip rkbin, fips, etc. for fast(er) development armbian-next: squash some `warn` loggings that were undue; remove `$SUDO_USER` handling in prepare-host.sh armbian-next: fix all `shellcheck` `errors` in `lib/` armbian-next: `shellcheck` downloader/runner; only for `lib/` armbian-next: disable `cpufrequtils` by default; to enable set `CPUFREQUTILS_ENABLE=true` [HIGHLY CONTENTIOUS CHANGE] armbian-next: fixes for interactive config; change wording of stuff and sort the list of boards in "expert" mode armbian-next: `docker`: fix when using `buildx`, always pass `--load` so produced image is actually available to Docker proper armbian-next: a few fixes (shortcircuits...) and enhancements for the GUI building; thanks NicoD armbian-next: introduce `SKIP_ARMBIAN_ROOTFS_CACHE` which is `=yes` by default armbian-next: introduce internal base tmp dir; check it early for sanity ref `noexec` and `nodev` mount options armbian-next: Python patching delusion, pt1 & pt2 & pt3 armbian-next: introduce ORAS-based kernel git bare tree seeding/bundles armbian-next: Python patching delusion, pt4: series & better markdown armbian-next: Python patching delusion pt 5 + EXTRAWIFI patch generator/harness + consistent patch mtime targets armbian-next: make sure at least one board config file is sourced, otherwise misspell `BOARD=` is hard to find armbian-next: `docker`: don't try to double init when recovering from failed pull armbian-next: bring `find_toolchain()` back from the dead; force `SKIP_EXTERNAL_TOOLCHAINS=no` to force use external toolchains armbian-next: fix `sunxi`/`sunxi64` using wrong KERNELPATCHDIR and incurring in old symlinks armbian-next: Revert "Fix issues 4589 (#4614)" (installing HEADERS dependencies manually) armbian-next: remove spurious `output.log` redirections in downloads.sh; `warn` that `download_and_verify()` is unhandled in armbian-next armbian-next: `git worktree`: forcefully fix the worktree's reference to the bare repo, and vice-versa armbian-next: ignore git errors due to missing 'origin' remote and/or revisions of `${SRC}/.git` during main-config armbian-next: git: setup the `worktree` first before trying to ensure safe armbian-next: drop old code for patching & fasthash armbian-next: better logging for `jethub`'s tweaks armbian-next: **regen library** after some 13 squashes (and repo.sh removal) armbian-next: aggregation: fix `PACKAGE_LIST_BOARD_REMOVE`/`PACKAGE_LIST_FAMILY_REMOVE`; also remove pkgs from DESKTOP aggregation armbian-next: `exec` into `sudo` when relaunching (AR-1452) armbian-next: validate cmdline params against malformed armbian-next: `reset_uid_owner` for the markdown log file too armbian-next: remove `@TODO` for stuff already done armbian-next: `git`: use `--recurse-submodules=no` for fetch; `fetch_from_repo()` handles submodules separately armbian-next: refactor `losetup -d ...` into `free_loop_device_insistent()` which retries 10 times armbian-next: `git`: mark submodule's dirs as safe before recursive `fetch_from_repo()` armbian-next: Python patching: pass `$HOME`/`$PATH` to Python, so git can find `.gitconfig` and Python can find `git` armbian-next: add (trap) cleanup handler to reset `./tmp` and `./output` to original pre-sudo UID before exiting, even with failure/abort armbian-next: `json-info`: do not replace ${SRC} armbian-next: fix: only aggregate packages if building images (AR-1471) armbian-next: only put `systemd-resolved`'s symlink in place if _not_ using NetworkManager armbian-next: Python tools (aggregation/patching): use separate pycache dir in ./cache; use unbuffered stdout/stderr armbian-next: tune logging in multiple places, there was a lot of redundant/un-needed info's armbian-next: better logging in `distro-agnostic.sh` armbian-next: remove very old extlinux killswitch armbian-next: happy new year, bump Igor's copyright armbian-next: `grub`: make `/usr/share/desktop-base/grub_background.sh` executable, otherwise does nothing armbian-next: `drivers-harness`: remove commit-date, do proper logging & escaping for git add/commit/format-patch armbian-next: `drivers-harness`: include the revert-IPX patch as a "driver" before all others armbian-next: `mvebu.conf::family_tweaks()`: small armbian-next'ify for better logging armbian-next: clearfog/cubox/odroidxu4: patching Makefile `armv5` to `armv7-a` so uboot compiles with modern gcc armbian-next: `rockchip-rk3588` `edge` bump & armbian-next'ify armbian-next: remove spurious debug logging with no useful info "Error-related files found" WiP: try for working armhf kernel-headers. leading nowhere. move to kernel-source / make modules_prepare later. WiP: kernel-headers via full source + `make modules_prepare` for 4.19 kernels. EXPERIMENTAL armbian-next: move compilation vars (chtreads, ccache stuff) from config to compile armbian-next: move var `HOSTRELEASE` from config to `prepare_host()` armbian-next: delete old aggregation code from config armbian-next: call hook `post_aggregate_packages()` only after actually aggregating armbian-next: avoid `linux-version` during configuration; `declare -g` instead of `export` armbian-next: config: reduce bash scope & remove some vars un-needed outside of config armbian-next: `cli`: `config-dump`: can now run on any bash/coreutils-enabled machine; remove some useless vars from dump armbian-next: `cli`: `json-info`: make errors clearer; don't include logs on success; small fixes armbian-next: `drivers-harness`: pass HOME/PATH explicitly to git commit/format-patch armbian-next: `drivers-harness`: make sure cached patch is valid; write it first to tmp file; redirect inside subshell armbian-next: split of `config-prepare` into multiple functions, allowing for logging of all non-interactive sections armbian-next: [focal-host] try to support `focal`-like host OS's again, via `python3.9` (and drop `buster` completely) armbian-next: [focal-host] try to support `focal`-like host OS's again, by avoiding git's `--no-auto-maintenance` armbian-next: [focal-host] remove usages of `declare -I`, not supported under focal's `bash` armbian-next: [focal-host] run `pip3` under the `python3_binary_path` (so coerce it to use 3.9 under focal) armbian-next: [focal-host] don't use git format-patch's `--no-encode-email-headers` which is not supported on focal's git armbian-next: [focal-host] don't use git's `--initial-branch` which is not supported on focal's git armbian-next: move aggregation to after prepare_host(), and thus out of config, and with correct Python deps armbian-next: patching: to-git: don't fail if is_autogen_dir; only change Subject if not rewriting in place armbian-next: patching: add sanity checks for magic markers matching mbox len & patch_contents not containing magic markers armbian-next: patching: go through parsing & splitting all patches before stopping if there are critical errors armbian-next: copy modified config + defconfig back from where it came from when `KERNEL_CONFIGURE=yes` armbian-next: uefi-common: `declare -g`, not `export` armbian-next: patching: don't gag if empty hexsha / no archeology possible armbian-next: patching: patches-to-git: don't eat newline between subject and message armbian-next: patching: grouping tryout, very basic by-dir: `SPLIT_PATCHES=yes` armbian-next: patching: use `--zero-commit` instead of find/replace for rewrite armbian-next: further split of `config-prepare()` and `do_main_configuration()` into `do_extra_configuration()` - fixes interactive desktop armbian-next: `SYNC_CLOCK`: don't attempt if `ntpd` is running; don't fail if `ntpdate` fails for any reason armbian-next: rootfs: don't stop if `df -h` fails armbian-next: `drivers-harness`: force `-c commit.gpgsign=false` during `git commit` Docs: slides Docs: slides (updated 28/Dec/2022) Docs: slides (updated 11/Jan/2023) armbian-next: `CLEAN_LEVEL=make-kernel` now does `git clean -xfd` instead, faster and 100% clean armbian-next: better logging for early apt installs during `prepare_host_basic()` armbian-oleg: handle error during host deps installation (Oleg has a mangled sources.list?) armbian-oleg: introduce `DOCKER_SIMULATE_CLEAN=yes` so I can pretend to be Oleg, but using Docker armbian-oleg: split hostdeps again, full of ifs, and reduce the minimum set of pkgs for Oleg-conditions while trying to keep Docker full armbian-next: `apt-cacher-ng` is now optional and activated via `MANAGE_ACNG=yes`; drop `NO_APT_CACHER` armbian-next: remove old template Dockerfile, -next does not use this at all armbian-oleg: `junk`: drastically reduce host-side dependencies / "remove junk" armbian-next: curb down wget logging for ORAS tooling download armbian-next: curb down `git fetch` logging; be verbose if user on terminal & not logging armbian-next: curb `aria2c`'s complaining a bit armbian-next: curb `free_loop_device_retried()`'s logging on first try armbian-next: long overdue split of `logging.sh` armbian-oleg: logging: new logfile format; `SHOW_LOG=yes` default; introduce `DEBUG=yes` armbian-next: fix quoting of retry var (cosmetic) armbian-oleg: better logs; only include non-empty .log files armbian-oleg: curb `apt-get`'s logging with `-q` / `-qq` armbian-oleg: curb Python launcher logs armbian-oleg: curb `rsync` logging during bsp armbian-oleg: drastically curb `update-initramfs` logging, unless SHOW_DEBUG=yes armbian-oleg: drastically curb `boot_logo()` logging armbian-oleg: drastically curb `apt-get` logging when running in chroot armbian-oleg: make `chroot_sdcard_apt_get_install_dry_run()` logging more useful by filtering the noise out armbian-oleg: if commands fail, write to log in bright red armbian-next: split `compilation/debs.sh`; `compile_xilinx_bootgen()` moved to family armbian-oleg: curb logging from building `armbian-firmware` armbian-oleg: curb logging from building `armbian-plymouth-theme`; do it in a logging section, like the others armbian-next: logging: reinit logging after processing cmdline parameters armbian-next: cli: half-assed, legacy based, `kernel` and `u-boot` cli commands armbian-next: `odroidxu4` vs kernel: move firmware hack to family using extension hook armbian-oleg: logging: drastically curb / make more useful kernel packaging logging armbian-oleg: logging: disable debugging (set -x) of generated postinst/etc kernel scripts (board-side, but also chroot logging) armbian-next: kernel: deterministic `.config` mtime handling; kernel build logging sections reorg armbian-next: remove `REPO_STORAGE` and `REPO_CONFIG` -- unused armbian-next: consider "Provided" installed packages when determining what is missing from host-side dependencies armbian-next: completely remove `acl` (which provided `getfacl`) and it's usages (basic deps) armbian-next: don't copy the host's `keyrings` to image armbian-next: Python tooling: use consolidated+hashed+cached pip base/pycache; don't pip during Dockerfile build, nor cli-requirements armbian-next: add core extension `c-plus-plus-compiler` which hostdeps on `g++-aarch64-linux-gnu` and enable it in JetHub family armbian-next: core extensions for `xfs` / `f2fs` / `brtfs` / `LUKS/cryptroot` hostdeps; enable in main-config armbian-next: Docker: abstract away and only run "docker info" once, it's very expensive armbian-next: `rootfs`: bunch'o'fixes, introduce `disable_systemd_service_sdcard()` to stop repeating incantantion armbian-next: don't warn about using extlinux armbian-next: a bunch of checks for Darwin; use brew's GNU coreutils first in PATH, so we get the correct utils; give instructions armbian-next: better logging for `kernel_prepare_bare_repo_from_oras_gitball()`; remove dead code armbian-next: fix re-launching of CLI commands under sudo (Docker was ok) armbian-next: `patching`: don't complain about missing files when the patch parsed OK and file was actually deleted armbian-next: `patching`: don't rewrite empty desc as "None" armbian-next: `patching`: remove quotes from author name during parsing armbian-next: `patching`: avoid mangling valid utf-8 from mbox'es armbian-next: `patching`: parse renamed source files, don't swallow them during rewrite armbian-next: optimize `config_source_board_file()`, now 600x faster by not reusing (slow) code from interactive armbian-next: optimize for `CONFIG_DEFS_ONLY=yes`, skipping stacktraces/git info armbian-next: introduce `POOR_MAN_PROFILER=yes` (under `ANSI_COLOR=none`) armbian-next: `docker`: fix Dockerfile output indentation & warning msgs about generation armbian-next: make sure temporary `kernel` and `u-boot` CLI commands always build a kernel/u-boot even if cached armbian-next: debug note about fstab with tmpfs in chroot armbian-next: mark Vagrant CLI as unimplemented armbian-next: `docker`: use dash-lines before/after launching docker; set `ARMBIAN_INSIDE_DOCKERFILE_BUILD=yes` when it is so armbian-next: `traps`: allow for cleanup handlers with arguments; unify around `run_one_cleanup_handler()` armbian-next: `ccache`: show more in ccache debugging armbian-next: make superglobals readonly (`DEST/WORKDIR/LOGDIR/SDCARD/MOUNT` and others) armbian-next: introduce `tmpfs-utils.sh`; put `LOGDIR` and `WORKDIR` under tmpfs; set `CCACHE_TEMPDIR` under `WORKDIR` armbian-next: split `prepare_host()`, fix `.tmp` reset trap armbian-next: do basic host checks before config if interactive, or during prepare-host if not armbian-next: tmpfs-utils: last-resort debugging, to stderr, of failed tmpfs unmount armbian-next: logging: squash nasty leaking file descriptor bug due to "tee" armbian-next: bring back `swig` hostdep -- needed for some u-boot's armbian-next: severe bug, countdown counted up armbian-next: kernel: cleanup bundle after patching succeeded, not after build success armbian-next: firmware: don't build `-full` firmware if not on CI/noninteractive and board's not going to use it armbian-next: `chroot_sdcard_apt_get_update()` to replace and better log `chroot_sdcard_apt_get update` armbian-next: extensions: rename `kernel-localmodconfig` to just `lsmod` armbian-next: docker: remove old code armbian-next: try harder to get `en_US.UTF-8` logging armbian-next: drop dead/duplicated code for bootsplash and mark kernel-drivers.sh (which includes bootsplash) as dead code armbian-next: extract `move_images_to_final_destination()` and optimize for fast-move when src/dst on same filesystem armbian-next: tune logging: unmounting, u-boot prep/source/install; traps; extensions armbian-next: don't leak `.raw` image if failure before move; show rootfs-tmpfs info armbian-next: `mktemp` and `tmpfs` related utility functions with automatic cleanup handlers armbian-next: small cleanups: squash typos / add function keyword / add types / mark dead code armbian-next: small cleanups: squash typos / add function keyword / add types / mark dead code armbian-next: add logging with size of actual built rootfs in MiB; debug tmpfs in trap & after pkgs done armbian-next: still fighting `tee` leaking under duress, and I think I won armbian-next: `compile_firmware()`: use new temp dir helpers (saves 2Gb+ in WORKDIR), fix typos armbian-next: severe bug, there was no prefix-emoji for Windows armbian-next: allow shortcut `EXT=` to the overly-long and plural `ENABLE_EXTENSIONS=` armbian-next: use retries for downloading ORAS tooling armbian-next: aggregation.py vs create-cache.sh: better hashing, much desktop armbian-next: patching.py vs decent logging: better, but not there yet Docs: slides (not-updated 18/Jan/2022) armbian-next: nextify "Re-enabling support for Rpi3" armbian-next: still fighting `tee` leaking under duress, turns out I _hadn't_ really won armbian-next: still fighting `tee` leaking: now I think I won, again armbian-next: somewhat-better handling/logging when using Docker (don't show two logfiles to inspect) armbian-next: rename too-long main function to `prep_conf_main_build_single()` armbian-next: hopefully improved error logging; avoid logging errors up to 3x; subdued red stacktraces armbian-next: `patching`: less horrible patching log, pt 37 armbian-next: fix missing `BOARD_TYPE` armbian-next: `git`: show "fetch completed" info, lest user thinks fetch is hanging while checking out armbian-next: `patching`: less horrible patching log, pt 38: downgrade utf-8 mbox to log warning, mark as problem armbian-next: fix ATF build under binutils 2.39+ via `TF_LDFLAGS=--no-warn-rwx-segment` armbian-next: add Debian Ports keyring hostdep to strap Debian riscv64 from ports armbian-next: `countdown`: split off file for countdown functions (countdown-to-abort and countdown-to-continue) armbian-next: `kernel`: `git`: split via-ORAS and via-Bundle tangle armbian-next: `kernel`: `git`: remove dead code, `GIT_PRE_FETCH_HOOK` / `checked_out_revision_mtime` armbian-next: replace bare `sync` invocations by introducing `wait_for_disk_sync()`; sync early at start of build armbian-next: adaptive shallow/full Kernel git bare git (3gb -> 300mb download) armbian-next: fix, don't try to get pid descendants on non-Linux (fixes Darwin cleanups) armbian-next: darwin: try to get rid of `.DS_Store` sneaky bastards before launching Docker armbian-next: back to cleaning downloaded bundles/gitballs after they've converted to bare tree & proven working armbian-next: git: detect, and stop, when git bare trees vs worktrees have been shuffled around and can't be recovered armbian-next: git: disallow forcing impossible shallow/full; warn if it's the case armbian-next: docker: archive logfiles again; warn & greet testers when archiving many old logfiles armbian-next: logging: new log file / summary nomenclature armbian-next: bsp: use `prepare_temp_dir_in_workdir_and_schedule_cleanup()`; bring back `PACKAGES_SEARCH_ROOT_ABSOLUTE_DIRS` armbian-next: firmware-deb.sh: prefix temp dir so it's "deb-firmware" armbian-next: extensions: move 'extensions.sh' into library; replace manual source with init function armbian-next: fix ANSI logs, use reset at strategic places armbian-next: `docker`: don't bomb if stdin is not a terminal (eg, in CI) armbian-next: `docker`: pass `CI` env down to Docker (for GHA etc) armbian-next: python/pip: solve crazy with `/usr/bin/pip3` by not using it (instead, `python3 -m pip`); don't mask errors armbian-next: `docker`: warn & countdown if we can't write to `cache/docker`; continue by using `.tmp/docker` if not aborted. armbian-next: countdown: if to test `-t 0` (0: stdin, 1:stdout) for interactive-ness; also test in if-not-aborted version armbian-next: interactive: info msg before (slow) gathering of all boards armbian-next: logging: (very, very early beta) for uploading/sharing of ANSI log file online armbian-next: kernel/logging: stop lying about LD/AR/INSTALL/SIGN/XZ make output armbian-next: patching: accept b4's magic marker `git@z Thu Jan 1 00:00:00 1970` for mbox validations armbian-next: mark `remove_all_trap_handlers()` deprecated, it never did anything armbian-next: downloads.sh: some cleanup and add debugging to `get_urls()` armbian-next: `logging`: validate where not already in a logging section when trying to start a new one armbian-next: `apt`: expand local caching to include package lists, as well as .deb's armbian-next: `main`: split `default-build` into `build-packages`, `start-end` armbian-next: `rootfs`: split `create-cache` into `rootfs-create` armbian-next: `rootfs`: new CLI command for rootfs cache building; further refactorings armbian-next: `rootfs`: adapt to new reality after huge refactor armbian-next: `aggregation`: dont' require or consider `BOARD` or `LINUXFAMILY` for aggregation anymore armbian-next: `rootfs`: build rootfs from CLI, sans-BOARD, sans-FAMILY, etc -- rootfs are meant to be shared armbian-next: `rootfs`: reset UID of produced rootfs tarball; announce what is doing and what has done armbian-next: `rootfs`: converge `rootfs-cache` and `rootfs` together; exchange BOARD for ARCH armbian-next: use unified methods for tmp dir handling all around (except ATF, which is special) armbian-next: extensions: `nomod`: for when you really don't care about modules armbian-next: we don't really need target `RELEASE` defined to install host deps (for now), don't freak out about it armbian-next: kernel-debs: re-enable postinst debugging; fix bad builddeb port for zImage armhf (image_name == NAME_KERNEL) armbian-next: cleanup, kill and add to-do's, remove dead code, turn down logging, squash future shortcircuit bugs armbian-next: extrawifi: include `driver_uwe5622_allwinner()` in harness armbian-next: extrawifi: include all `patch/misc/wireless*.patch` and `patch/misc/*wifi*.patch` in drivers hash calculation armbian-next: kernel-debs: force `arch/arm/boot/zImage` as vmlinuz for armhf armbian-next: kernel-debs: squash some warnings; add `KERNEL_SRC_ARCH` to arch configs; don't warn about `NAME_KERNEL` armbian-next: wait_for_disk_sync(): warn only if more than 30s fsync armbian-next: binfmt: make sure we have all `binfmt`'s setup & enabled for `rootfs` build (v2) armbian-next: `docker`: don't spew errors about sharing cache if cache doesn't even exist armbian-next: GHA & rootfs adventures, pt 1 armbian-next: aggregation: quick & dirty "potential paths" in Markdown armbian-next: cli: `kernel` cli forces `KERNEL_CONFIGURE=no`; @TODO this should only "default to", not force. for proper kernel cli, later armbian-next: logging: pure ASCII .log; included in Markdown for GHA armbian-next: docker/gha: pass down a bunch of GHA specific vars (list unconfirmed) armbian-next: rootfs: curb logging; GHA outputs `rootfs_cache_id_version`, `rootfs_cache_id` and `rootfs_out_filename_relative` armbian-next: ext: `grub-riscv64`: syncs with original version, fixes, nextify armbian-next: ext: `grub-sbc-media`: syncs with original version, fixes, nextify Docs: slides (1-slide update 25/Jan/2023) armbian-next: drop buildpkg code armbian-next: patching: rewrite: don't gag on bad "/dev/null.c" rename patches; archeology: `--find-copies-harder`; try to get GHA to run GH pages armbian-next: chroot_sdcard_apt_get_install_dry_run(): run again if first try fails, with more logging armbian-next: `docker`: passthrough of the SSH agent, via Docker magic on Darwin; enable for `docker-shell` and `kernel-patches-to-git` armbian-next: `docker`: include `${SRC}/.git` static via Dockerfile, for `kernel-patches-to-git` / archeology; use fast archeology armbian-next: all chroots: mount `/var/tmp` as tmpfs; create dir `/run/user/0` and mount as tmpfs @TODO cleanups armbian-next: docker: pass `ARMBIAN_ENABLE_CALL_TRACING=yes` via env, and only if `DOCKER_ARMBIAN_ENABLE_CALL_TRACING=yes` armbian-next: compress-checksum: move config to config-prepare; drop weight; kill warnings armbian-next: repeat build opts: fix missing BOARD, show both after config (for logfile) & after build armbian-next: bump logging archive greet to 14 files armbian-next: fix: kernel-git-oras: make sure `cache_git_bare_dir` exists before trying to decide which media it's on armbian-next: git: `fetch_from_repo()`: add fetched_revision(_ts) as well as checked_out_revision(_ts); introduce param `do_checkout=no` to only fetch, not checkout armbian-next: cli-build: don't show two `BOARD=` repeat option armbian-next: aggregate_packages out of `main_default_start_build()`; introduce `assert_requires_aggregation()` armbian-next: firmware-deb: reworked: Provides/Conflicts instead of Replaces; `cp` -> `git archive`; new firmware CLI; FULL version .deb out of tmpfs armbian-next: unify 2 methods for config: `prep_conf_main_minimal_ni()` (rootfs/firmware/kernel) and `prep_conf_main_build_single()` (full-build, possibly interactive) armbian-next: curb some logging for git non-checkouts armbian-next: prepare-host: don't manage ACNG in prepare if not asked to armbian-next: docker: fix: not all `find`'s are made equal, back to `-mtime +1` (full 24-hour period needed for match) armbian-next: interactive: review `-t 0` instead of `-t 1`; move non-interactive `check_basic_host()` from prepare-host to config-prepare, for consistency armbian-next: move aggregation-had-run check to more strategic places (pre-bsp, pre-rootfs) instead of at the beginning of full_build_packages_rootfs_and_image() armbian-next: docker: split `docker_cli_build_dockerfile()` into `docker_cli_prepare_dockerfile()`; introduce `FAST_DOCKER` armbian-next: git: refactor `git_parse_ref()` out of `fetch_from_repo()` for reuse armbian-next: config: twist `prep_conf_main_minimal_ni()` to receive `use_board=` armbian-next: do-not-fix docker-asking-for-docker error by adding a "tip" armbian-next: cli: *do not* allow running as root on terminal; make `PREFER_DOCKER=no` actually work armbian-next: cli: try (and fail, probably...) to be helpful with half-working Docker installs WiP: artifacts: kernel - pt1: versioning WiP: artifacts: kernel - pt2: memoizing git ref2info WiP: artifacts: kernel - pt3: drivers+patches+.config hashing WiP: artifacts: kernel - pt4: squash unrelated bugs that show up; move `prepare_compilation_vars()` to default build WiP: artifacts: kernel - pt5: tune kernel version, refactor WiP: artifacts: u-boot - pt6: add artifact for u-boot WiP: artifacts: u-boot/kernel - pt7 - adapt legacy/artifact versions; use common `capture_rename_legacy_debs_into_artifacts()` armbian-next: logging: detect `TERM=alacritty` and use normal instead of gray armbian-next: fix: for the 1000th time had broken `KERNEL_ONLY=yes`; curb some warnings armbian-next: introduce `assert_prepared_host()` and assert before using any Python, to un-mask any further fallacies armbian-next: take a breath: config might mark aggregation required; aggregation requires prepare_host, and thus runs in default-build armbian-next: logging: ansi-haste sharing beta 2 (now with URL) armbian-next: docker: fix Dockerfile generation; don't output logs info during Dockerfile build armbian-next: u-boot: downgrade more errors to warnings, so we can build some legacy u-boot's armbian-next: cli: u-boot alias: don't stop to interactively ask for kernel configuration during u-boot build armbian-next: call-stack-analyze.py: detect missing function_to_file and stop properly (half-finished traces?) armbian-next: json-info: cleanup info/json/csv/opensearch, add logging, add some very basic instructions; add Kibana dashboards & docker-compose to bring it OS+Kibana up armbian-next: shellcheck: update, cleanup script, invent some new severities armbian-next: shellcheck: squash a bunch of 'critical' warnings armbian-next: shellcheck: raise the bar, stop excluding "SC2155 Declare and assign separately to avoid masking return values" armbian-next: squash some shellcheck warnings; don't `local` and assign subshell value in same statement, that masks errors (`declare`/`local` almost-always works) armbian-next: shellcheck: raise the bar, stop excluding "SC2154 is referenced but not assigned" armbian-next: shellcheck: squash more shellcheck warnings; some long-lost variables being used; some unset's that are better reset's, etc armbian-next: `docker-shell`: provide SSH and forward ssh auth socket to `docker-shell` armbian-next: docker: pass `CARD_DEVICE` to Docker via `--device` armbian-next: u-boot: mark bare repo as safe before using it (@TODO `git_ensure_safe_directory()` stuff, again) armbian-next: swap github in as default u-boot mirror; allow `denx` to use upstream's gitlab armbian-next: git ref2info: dereference annotated tags to get "real" commit SHA1; fail if curl failed; validate parse_makefile_version() armbian-next: hash/cache: remove some cruft, some better logging armbian-next: docker: pass down git info via env var, since Docker doesn't get `${SRC}/.git`, yet we need that info in the Docker logs armbian-next: u-boot: fix: check gcc version before trying to downgrade certain errors to warnings [kinetic] un-symlink `cli`/`main` `kinetic`->`jammy` packages; add `systemd-resolved` which was split off from main `systemd` package in Kinetic `risc-v`/`starfive`: new `starfive` family with their (vendor) kernel `risc-v`/`starfive`: use mainline kernel 6.1.y, with StarFive's rebased patches against `v6.1.5` `risc-v`/`starfive`: `CONFIG_MOTORCOMM_PHY=m` for the onboard Ethernet `odroidc2`: u-boot: use minimal patchset for v22.01 u-boot `meson64`: edge/6.2: spi-nor: add support for XT25Q64 in addition to XT25F128B `meson64`: edge/6.2: bump to `6.2-rc5` `meson64`: edge/6.2: rebase onto `v6.2-rc5` `odroidm1`: enable working userland `fw_setenv`, so users can go back to Petitboot & otherwise tweak u-boot env from Armbian `odroidm1`: bump to 6.2-rc3; look ma, no patches. `media`: fixes for bad mbox/unneeded patches `media-edge` and `media-current` - thanks to @schwar3kat `sunxi/sunxi64`: disable some sunxi/sunxi64 "readme" patches that contain other patches -- for discussion `rk322x-box`: fix relative path reference to .bin in patch (a-n changed u-boot path on disk) - for @paolosabatino `rockchip64`: current/5.15: remove IPX revert patch that's already in drivers now `rockchip64`: current/5.15: fix bad unidiff/lack of mbox of wifi-hf-lps170-bl602.patch & makefile odroidm1: edge: bump to `6.2-rc5` `sun50iw9`: half-fixes to get going under armbian-next (@TODO needs review/nextify/confirm busybox is not needed) WiP: bump edge to `v6.2-rc6` for UEFI / meson64 / rk3568-odroid / rockchip-rk3588 `meson64`: edge/6.2: arm64: dts: amlogic: meson-sm1-odroid-hc4: fix active fan thermal trip WiP: `odroidhc4`: u-boot to 23.01; add HC4-SPI target preboot for USB; change HC4-SD target defconfig to new `hc4_sd`, similar to `c4` Revert "Add some Wireless modules to RK3588 legacy kernel (#4759)" Docs: slides (2-slide update 01/Feb/2023) Igor <igor@armbian.com> (2): armbian-next: cli: add CLI alias command `rootfs-cache` for creating rootfs cache armbian-next: rootfs: introduce `ROOTFS_COMPRESSION_RATIO`, default 5 for general use, default 15 for `rootfs` CLI; fix pv | tar bug; remove GPG signing Lane Jennison <lane@lane-fu.com> (1): armbian-next: fix old fuction name refrence cleaning to general_cleaning Richard Neese <r.neese@gmail.com> (1): `risc-v`/`starfive`: switch from `grub` to `extlinux` Tony <tonymckahan@gmail.com> (1): armbian-next: officially support WSL2; pester user for UTF-8 terminal amazingfate <liujianfeng1994@gmail.com> (1): `rockchip64`: update bl31 and ddr blobs of rk3568 [cherry-pick]

git shortlog since last week... Ricardo Pardini <ricardo@pardini.net> (74): armbian-next: nextify "Re-enabling support for Rpi3" armbian-next: still fighting `tee` leaking under duress, turns out I _hadn't_ really won armbian-next: still fighting `tee` leaking: now I think I won, again armbian-next: somewhat-better handling/logging when using Docker (don't show two logfiles to inspect) armbian-next: rename too-long main function to `prep_conf_main_build_single()` armbian-next: hopefully improved error logging; avoid logging errors up to 3x; subdued red stacktraces armbian-next: `patching`: less horrible patching log, pt 37 armbian-next: fix missing `BOARD_TYPE` armbian-next: `git`: show "fetch completed" info, lest user thinks fetch is hanging while checking out armbian-next: `patching`: less horrible patching log, pt 38: downgrade utf-8 mbox to log warning, mark as problem armbian-next: fix ATF build under binutils 2.39+ via `TF_LDFLAGS=--no-warn-rwx-segment` `media`: fixes for bad mbox/unneeded patches `media-edge` and `media-current` - thanks to @schwar3kat `rk322x-box`: fix relative path reference to .bin in patch (a-n changed u-boot path on disk) - for @paolosabatino armbian-next: `countdown`: split off file for countdown functions (countdown-to-abort and countdown-to-continue) armbian-next: `kernel`: `git`: split via-ORAS and via-Bundle tangle armbian-next: `kernel`: `git`: remove dead code, `GIT_PRE_FETCH_HOOK` / `checked_out_revision_mtime` armbian-next: replace bare `sync` invocations by introducing `wait_for_disk_sync()`; sync early at start of build WiP: adaptive shallow/full Kernel git bare git (3gb -> 300mb download) armbian-next: fix, don't try to get pid descendants on non-Linux (fixes Darwin cleanups) armbian-next: darwin: try to get rid of `.DS_Store` sneaky bastards before launching Docker armbian-next: back to cleaning downloaded bundles/gitballs after they've converted to bare tree & proven working armbian-next: git: detect, and stop, when git bare trees vs worktrees have been shuffled around and can't be recovered armbian-next: git: disallow forcing impossible shallow/full; warn if it's the case armbian-next: docker: archive logfiles again; warn & greet testers when archiving many old logfiles armbian-next: logging: new log file / summary nomenclature armbian-next: bsp: use `prepare_temp_dir_in_workdir_and_schedule_cleanup()`; bring back `PACKAGES_SEARCH_ROOT_ABSOLUTE_DIRS` armbian-next: firmware-deb.sh: prefix temp dir so it's "deb-firmware" armbian-next: extensions: move 'extensions.sh' into library; replace manual source with init function armbian-next: fix ANSI logs, use reset at strategic places armbian-next: `docker`: don't bomb if stdin is not a terminal (eg, in CI) armbian-next: `docker`: pass `CI` env down to Docker (for GHA etc) armbian-next: python/pip: solve crazy with `/usr/bin/pip3` by not using it (instead, `python3 -m pip`); don't mask errors armbian-next: `docker`: warn & countdown if we can't write to `cache/docker`; continue by using `.tmp/docker` if not aborted. armbian-next: countdown: if to test `-t 0` (0: stdin, 1:stdout) for interactive-ness; also test in if-not-aborted version armbian-next: interactive: info msg before (slow) gathering of all boards armbian-next: logging: (very, very early beta) for uploading/sharing of ANSI log file online armbian-next: kernel/logging: stop lying about LD/AR/INSTALL/SIGN/XZ make output armbian-next: patching: accept b4's magic marker `git@z Thu Jan 1 00:00:00 1970` for mbox validations armbian-next: mark `remove_all_trap_handlers()` deprecated, it never did anything armbian-next: downloads.sh: some cleanup and add debugging to `get_urls()` armbian-next: `logging`: validate where not already in a logging section when trying to start a new one armbian-next: `apt`: expand local caching to include package lists, as well as .deb's armbian-next: `main`: split `default-build` into `build-packages`, `start-end` armbian-next: `rootfs`: split `create-cache` into `rootfs-create` armbian-next: `rootfs`: new CLI command for rootfs cache building; further refactorings armbian-next: `rootfs`: adapt to new reality after huge refactor armbian-next: `aggregation`: dont' require or consider `BOARD` or `LINUXFAMILY` for aggregation anymore armbian-next: `rootfs`: build rootfs from CLI, sans-BOARD, sans-FAMILY, etc -- rootfs are meant to be shared armbian-next: `rootfs`: reset UID of produced rootfs tarball; announce what is doing and what has done armbian-next: `rootfs`: converge `rootfs-cache` and `rootfs` together; exchange BOARD for ARCH armbian-next: use unified methods for tmp dir handling all around (except ATF, which is special) armbian-next: extensions: `nomod`: for when you really don't care about modules armbian-next: we don't really need target `RELEASE` defined to install host deps (for now), don't freak out about it armbian-next: kernel-debs: re-enable postinst debugging; fix bad builddeb port for zImage armhf (image_name == NAME_KERNEL) armbian-next: cleanup, kill and add to-do's, remove dead code, turn down logging, squash future shortcircuit bugs armbian-next: extrawifi: include `driver_uwe5622_allwinner()` in harness armbian-next: extrawifi: include all `patch/misc/wireless*.patch` and `patch/misc/*wifi*.patch` in drivers hash calculation armbian-next: kernel-debs: force `arch/arm/boot/zImage` as vmlinuz for armhf armbian-next: kernel-debs: squash some warnings; add `KERNEL_SRC_ARCH` to arch configs; don't warn about `NAME_KERNEL` armbian-next: wait_for_disk_sync(): warn only if more than 30s fsync armbian-next: binfmt: make sure we have all `binfmt`'s setup & enabled for `rootfs` build (v2) armbian-next: `docker`: don't spew errors about sharing cache if cache doesn't even exist armbian-next: GHA & rootfs adventures, pt 1 armbian-next: aggregation: quick & dirty "potential paths" in Markdown armbian-next: riscv64: move `debian-ports-archive-keyring` from main to debootstrap; also only for `sid` meson64: edge/6.2: spi-nor: add support for XT25Q64 in addition to XT25F128B meson64: edge/6.2: bump to `6.2-rc5` meson64: edge/6.2: rebase onto `v6.2-rc5` armbian-next: cli: `kernel` cli forces `KERNEL_CONFIGURE=no`; @TODO this should only "default to", not force. for proper kernel cli, later armbian-next: logging: pure ASCII .log; included in Markdown for GHA armbian-next: docker/gha: pass down a bunch of GHA specific vars (list unconfirmed) armbian-next: rootfs: curb logging; GHA outputs `rootfs_cache_id_version`, `rootfs_cache_id` and `rootfs_out_filename_relative` Docs: slides (1-slide update 25/Jan/2022) ------------------------ armbian-next END marker ----------------------------- Igor <igor@armbian.com> (3): Add CLI alias command `rootfs-cache` for creating rootfs cache armbian-next: rootfs: introduce `ROOTFS_COMPRESSION_RATIO`, default 5 for general use, default 15 for `rootfs` CLI; fix pv | tar bug; remove GPG signing Clean and unify XFCE and Cinnamon (squashed)

Since I've no slides, a terrible sense of humor, and nothing else to post, here's a git shortlog of the changes since last week. Ricardo Pardini <ricardo@pardini.net> (99): armbian-next: `CLEAN_LEVEL=make-kernel` now does `git clean -xfd` instead, faster and 100% clean armbian-next: better logging for early apt installs during `prepare_host_basic()` armbian-oleg: handle error during host deps installation (Oleg has a mangled sources.list?) armbian-oleg: introduce `DOCKER_SIMULATE_CLEAN=yes` so I can pretend to be Oleg, but using Docker armbian-oleg: split hostdeps again, full of ifs, and reduce the minimum set of pkgs for Oleg-conditions while trying to keep Docker full armbian-next: `apt-cacher-ng` is now optional and activated via `MANAGE_ACNG=yes`; drop `NO_APT_CACHER` armbian-next: remove old template Dockerfile, -next does not use this at all armbian-oleg: `junk`: drastically reduce host-side dependencies / "remove junk" armbian-next: curb down wget logging for ORAS tooling download armbian-next: curb down `git fetch` logging; be verbose if user on terminal & not logging armbian-next: curb `aria2c`'s complaining a bit armbian-next: curb `free_loop_device_retried()`'s logging on first try armbian-next: long overdue split of `logging.sh` armbian-oleg: logging: new logfile format; `SHOW_LOG=yes` default; introduce `DEBUG=yes` armbian-next: fix quoting of retry var (cosmetic) armbian-oleg: better logs; only include non-empty .log files armbian-oleg: curb `apt-get`'s logging with `-q` / `-qq` armbian-oleg: curb Python launcher logs armbian-oleg: curb `rsync` logging during bsp armbian-oleg: drastically curb `update-initramfs` logging, unless SHOW_DEBUG=yes armbian-oleg: drastically curb `boot_logo()` logging armbian-oleg: drastically curb `apt-get` logging when running in chroot armbian-oleg: make `chroot_sdcard_apt_get_install_dry_run()` logging more useful by filtering the noise out armbian-oleg: if commands fail, write to log in bright red armbian-next: split `compilation/debs.sh`; `compile_xilinx_bootgen()` moved to family armbian-oleg: curb logging from building `armbian-firmware` armbian-oleg: curb logging from building `armbian-plymouth-theme`; do it in a logging section, like the others armbian-next: logging: reinit logging after processing cmdline parameters armbian-next: cli: half-assed, legacy based, `kernel` and `u-boot` cli commands armbian-next: `odroidxu4` vs kernel: move firmware hack to family using extension hook armbian-oleg: logging: drastically curb / make more useful kernel packaging logging armbian-oleg: logging: disable debugging (set -x) of generated postinst/etc kernel scripts (board-side, but also chroot logging) armbian-next: kernel: deterministic `.config` mtime handling; kernel build logging sections reorg armbian-next: remove `REPO_STORAGE` and `REPO_CONFIG` -- unused armbian-next: consider "Provided" installed packages when determining what is missing from host-side dependencies armbian-next: completely remove `acl` (which provided `getfacl`) and it's usages (basic deps) armbian-next: don't copy the host's `keyrings` to image armbian-next: Python tooling: use consolidated+hashed+cached pip base/pycache; don't pip during Dockerfile build, nor cli-requirements armbian-next: add core extension `c-plus-plus-compiler` which hostdeps on `g++-aarch64-linux-gnu` and enable it in JetHub family armbian-next: core extensions for `xfs` / `f2fs` / `brtfs` / `LUKS/cryptroot` hostdeps; enable in main-config armbian-next: Docker: abstract away and only run "docker info" once, it's very expensive armbian-next: `rootfs`: bunch'o'fixes, introduce `disable_systemd_service_sdcard()` to stop repeating incantantion armbian-next: don't warn about using extlinux armbian-next: a bunch of checks for Darwin; use brew's GNU coreutils first in PATH, so we get the correct utils; give instructions armbian-next: better logging for `kernel_prepare_bare_repo_from_oras_gitball()`; remove dead code armbian-next: fix re-launching of CLI commands under sudo (Docker was ok) armbian-next: `patching`: don't complain about missing files when the patch parsed OK and file was actually deleted armbian-next: `patching`: don't rewrite empty desc as "None" armbian-next: `patching`: remove quotes from author name during parsing armbian-next: `patching`: avoid mangling valid utf-8 from mbox'es armbian-next: `patching`: parse renamed source files, don't swallow them during rewrite armbian-next: optimize `config_source_board_file()`, now 600x faster by not reusing (slow) code from interactive armbian-next: optimize for `CONFIG_DEFS_ONLY=yes`, skipping stacktraces/git info armbian-next: introduce `POOR_MAN_PROFILER=yes` (under `ANSI_COLOR=none`) armbian-next: `docker`: fix Dockerfile output indentation & warning msgs about generation armbian-next: make sure temporary `kernel` and `u-boot` CLI commands always build a kernel/u-boot even if cached `risc-v`: correctly include `debian-ports-archive-keyring` in Risc-V CLI packages `risc-v`/`starfive`: use mainline kernel 6.1.y, with StarFive's rebased patches against `v6.1.5` `risc-v`/`starfive`: update kernel config, sans changes `risc-v`/`starfive`: `CONFIG_MOTORCOMM_PHY=m` for the onboard Ethernet `risc-v`/`starfive`: switch kernel config to `savedefconfig`'s output, cos that's what it should be `odroidm1`: enable working userland `fw_setenv`, so users can go back to Petitboot & otherwise tweak u-boot env from Armbian `odroidm1`: bump to 6.2-rc3; look ma, no patches. `odroidhc4`: pre-configure `fancontrol` so fans work right out of the box armbian-next: debug note about fstab with tmpfs in chroot armbian-next: mark Vagrant CLI as unimplemented armbian-next: `docker`: use dash-lines before/after launching docker; set `ARMBIAN_INSIDE_DOCKERFILE_BUILD=yes` when it is so armbian-next: `traps`: allow for cleanup handlers with arguments; unify around `run_one_cleanup_handler()` armbian-next: `ccache`: show more in ccache debugging armbian-next: make superglobals readonly (`DEST/WORKDIR/LOGDIR/SDCARD/MOUNT` and others) armbian-next: introduce `tmpfs-utils.sh`; put `LOGDIR` and `WORKDIR` under tmpfs; set `CCACHE_TEMPDIR` under `WORKDIR` armbian-next: split `prepare_host()`, fix `.tmp` reset trap armbian-next: do basic host checks before config if interactive, or during prepare-host if not armbian-next: tmpfs-utils: last-resort debugging, to stderr, of failed tmpfs unmount armbian-next: logging: squash nasty leaking file descriptor bug due to "tee" armbian-next: bring back `swig` hostdep -- needed for some u-boot's armbian-next: severe bug, countdown counted up armbian-next: kernel: cleanup bundle after patching succeeded, not after build success armbian-next: firmware: don't build `-full` firmware if not on CI/noninteractive and board's not going to use it armbian-next: `chroot_sdcard_apt_get_update()` to replace and better log `chroot_sdcard_apt_get update` armbian-next: extensions: rename `kernel-localmodconfig` to just `lsmod` armbian-next: docker: remove old code armbian-next: try harder to get `en_US.UTF-8` logging armbian-next: drop dead/duplicated code for bootsplash and mark kernel-drivers.sh (which includes bootsplash) as dead code armbian-next: extract `move_images_to_final_destination()` and optimize for fast-move when src/dst on same filesystem armbian-next: tune logging: unmounting, u-boot prep/source/install; traps; extensions armbian-next: don't leak `.raw` image if failure before move; show rootfs-tmpfs info armbian-next: `mktemp` and `tmpfs` related utility functions with automatic cleanup handlers armbian-next: small cleanups: squash typos / add function keyword / add types / mark dead code armbian-next: small cleanups: squash typos / add function keyword / add types / mark dead code armbian-next: add logging with size of actual built rootfs in MiB; debug tmpfs in trap & after pkgs done armbian-next: still fighting `tee` leaking under duress, and I think I won armbian-next: `compile_firmware()`: use new temp dir helpers (saves 2Gb+ in WORKDIR), fix typos armbian-next: severe bug, there was no prefix-emoji for Windows armbian-next: allow shortcut `EXT=` to the overly-long and plural `ENABLE_EXTENSIONS=` armbian-next: use retries for downloading ORAS tooling WiP: aggregation.py vs create-cache.sh: better hashing, much desktop WiP: patching.py vs decent logging: better, but not there yet ------------------------ armbian-next END marker ----------------------------- Richard Neese <r.neese@gmail.com> (1): `risc-v`: switch from `grub` to `extlinux`; add Debian Ports keyring hostdep to strap Debian riscv64 from ports Tony <tonymckahan@gmail.com> (1): armbian-next: officially support WSL2; pester user for UTF-8 terminal

Err... no, current patching log is a terrible mess and very hard to tell wtf is going on. I am making a complete review of the patching logging. Objective is to produce a simple "table-like" output like we had before, on screen; then have details/rejects/etc on the logfile when investigating. Sorry for the bad experience, it's gonna be much better "real soon now". PS: For now, the least-horrible way to see the Patching info is looking at the produced .md file in output/logs. Render it to UTF-8 HTML for full glory.

A lie. I just found a bug where I mangled perfectly valid UTF-8, during mbox parsing, for no good reason. Fixing...

Yeah, parsing the patches includes being able to decode them. Patches should match the encoding of the target patched source; In this case, it's UTF-8. In my view, those patches are mangled (not just "in big5"): the original encoding was lost due to copy/pasting, and they're already (in master) putting frakked up characters in the source code. So it can't get worse, but now I warn you about the error.

It suits a lot of people -- not only you. I am (seriously) taking in the sensible parts of your feedback and implementing. (it would be easier if you were not also insulting, I get distracted by the insults when parsing your text). I already got most of the "junk" out ("junkless", for the "Oleg" scenario: fully native arm64 assembly, no Docker) while still keeping the capacity to have a "junkfull" Docker image with everything (without having to sync 2 pieces). It's _a lot_ of if's. Next, (pun!), I will handle the logging of the patch apply to output what we had before (just patch name, OK or not, colors), and hide the details in the logfile. (eg; rejects, etc). Then reduce the logging to file of things we don't need & fix the formatting of the single .txt log as has already been discussed a few pages back. Also https://armbian.atlassian.net/browse/AR-1451 so logs actually show up on screen. So it's just more code / more if's -- and more scenarios to test. The whole reason I wanted to avoid _adding more code_ is that the more code I add, the more code to review, and the longer it will take to merge. But I'd rather have everyone onboard, or, if that's not possible, have most people onboard, even if takes a bit longer.

Definitely not for Kernel. Consider a new user, that wants to patch something really simple, for example, add a missing LED to a board via DT. (that's how I got started.) What is easier / more productive: download kernel-source-xxx package, make it work somehow, see a complete source without any history/commits, not understand why things are done that way... etc. Or: go to armbian/build repo, see history & documentation & happy friendly people collaborating & a tool that guides them to produce a new build & a good place to PR his small patch so that it lives later....

Take a look at `patching.py` if you can muster, for context. I am trying to strike a difficult balance: if the file passes a few tests (matching magic markers, unidiff well-formed, others) then I handle it as an mbox, and split them. In certain conditions, as you mentioned, this fails, as is the case with heavily-mangled files, and the build refuses to continue, as an indication of "the bar has been raised, and xxxx patch doesn't pass". In any case, the actual patches are being applied using the `patch` utility like always (so allow for fuzzy/offset, etc), and if it turns out half-decent patches is "impossible" (it isn't...) the code can be made to not split, and will then behave just like before. We could do that per-family, especially for legacy stuff no-one wants to or will fix. So let's strive to clean -- it's really not that hard (or what am I missing?). Later, we can add GitHub PR-level automatic checks for well-formed patches, thus "keeping" it clean for the future.

Yes, I have already started work on `armbian-oleg` -- it only builds media, only on arm64, and does not log anything.

I see. Let's then focus 100% energy of the whole team to cater to Oleg's specific wishes and needs until he is satisfied. Also, let's give him his own family, let him arbitrary fuck up and duplicate code for no reason, enable Tegra on Rockchip family, etc etc, and then in the end complain. When does it stop?

Yes, only if amd64. Oleg is about "only arm64". I will thus stop all development and start endlessly bitching about "junk" and being useless.

You are delusional. `master` downloads & extracts 9gb of multiple different versions of Linaro etc toolchains before it eats any breakfast.

Awesome. I just pushed `8a2f30f4a1d648c4e15df5944241e80847bc8f78` with: Fixed; I check if the pidfile for ntpd is there, and if so, skip it completely, saving us 2 seconds. Even if not skipped, I added the "|| true" so it won't fail for any reason. https://github.com/armbian/build/commit/d42d340f48848126abb571b389b65d68a1732da4 Fixed (hopefully), I just added "|| true" -- there's work going on in master to avoid the double-mount and other bugs that might appear here, I'll wait for them. https://github.com/armbian/build/commit/e84d32785f8de4eab4763c120bd4c367d4ee6741 Added as a command-line option to the "git ... commit" invocation at that stage, so shouldn't fail anymore even if you have it enabled globally. https://github.com/armbian/build/commit/96926fbf4f8a1cb96dc18e4a4411f9fc3c594bec This was due to a borked merge coming from master, was up for a few hours, sorry about this. (ref extrawifi) Fixed. Another case of "let's ask the user" in the middle of configuration -- I had forgotten this. https://github.com/armbian/build/commit/1e53ac23a84c9135394dead7a0bf5ca8f8dfbf58 Please notice there's a known issue with desktop rootfs caching I am investigating. Super thanks @schwar3kat for all the feedback and patience.

Hmm. Yes. This is how buildd / debuild / Launchpad works. You create a source package (a .dsc, .changes file) including the pristine source in a .tar, then the "debian" packaging source. Then you build it with "debuild" or "dpkg-buildpackage". This would be great, but is very different from what we do now, won't support cross-compilation, and thus requiring native iron to build any package. It also requires release-specific packages, even for thing that aren't really release-specific (say, firmware). So yes, this is maybe for 2024.