-
Posts
132 -
Joined
-
Last visited
Content Type
Forums
Store
Crowdfunding
Applications
Events
Community Map
Posts posted by rpardini
-
-
22 hours ago, schwar3kat said:
I noticed a distcc.sh in the compilation folder. I'm guessing/hoping that this will mean easy distributed compilation.
Yep, I worked on that a few months ago. We've something like `./compile.sh distccd` which starts a distcc daemon, complete with zeroconf etc, using the same toolchains Armbian would.
Something else (forget what) enables it to be used on a controller node. You do need a bigger controller machine, say 8-cores, to completely max out some two/three 4-core distccd's, due to the high preprocessing load.
The network between the machines has to be very good, too (gigabit+). All in all is a "fun" way to use all those SBCs laying around doing nothing, but don't expect something miraculous...
One day I'll finish / document this...
-
On 2/3/2023 at 9:38 AM, schwar3kat said:
Example: orangepi-r1plus-lts.conf. I just changed some of the values in the two files that are written into the build and the changes don't make it into the build. The bsp cache is used.Yep, not only are changes to hooks ignored, but changes to core code are _also_ ignored. For now the only way to get around it is to remove the .deb in question from output/debs (or CLEAN_LEVEL=debs, although that cleans too much).
Meanwhile I'm working on almost exactly the same subject but for packages like kernel and u-boot, and bsps will come after that. Essentially I'll be hashing the bash code of drivers/patches/config + core+hooks and including that into the .deb Version: field.
-
On 2/5/2023 at 3:03 AM, TRS-80 said:
@rpardini, as you obviously know the work best, could you be so kind as to indulge me a summary of the main features? Not to discount your work, but for the average reader I think a 20 (50? more?) item long commit log does not make for the most interesting article.
Maybe we can hash something out in lanefu's HedgeDoc instance again, like that one time? I think that worked pretty well.
Yep, good idea. Commit log is at 400+ commits... I've slides (in Markdown/HedgeDoc) on my own instance, that I've copied to Lane's at https://docs.lane-fu.com/dAGOaCTLQ0SgPnQsHz6LWw?both -- it's in slide style, first few ones are outdated, but can serve as base: https://docs.lane-fu.com/dAGOaCTLQ0SgPnQsHz6LWw?both -- also, if you have patience to watch the developer meeting videos, most secrets will be revealed. I was hoping to do one last consolidated/updated presentation before the switch, but time is pressing...
-
here's the git shortlog; as mentioned in the meeting, I've reworded quite a bit, so this a lot longer (goes back in time further) than for other meetings.
Ricardo Pardini <ricardo@pardini.net> (441): armbian-next: `lib/tools` - python - config definitions capture and massaging armbian-next: `general-packaging`, `mkdebian` and `builddeb` are NO MORE armbian-next: update defaults and comments in config file armbian-next: atf: downgrade gcc errors to warnings, so ATF 2.5, 2.2 and others can be built on modern toolchains armbian-next: fix for .bin/.elf copy from atf to uboot dir armbian-next: kernel: config: `make olddefconfig` (not `oldconfig`, which is interactive) when `KERNEL_CONFIGURE=yes` armbian-next: kernel build: always add container (series/patch-dir) mtime as minimum mtime for patch results armbian-next: do not exclude 'CC' messages from kernel make armbian-next: remove warm/cold bundles from `fetch_from_repo`, replace with 2 invocation-specific hooks and `gitballs` (.tar of .git) armbian-next: try to parse `Source: linux-5.x.y-rcZ` correctly into `VER=5.x.y-rcZ` instead of `VER=5.x.y` armbian-next: allow overriding rkbins repo URL with `RKBIN_GIT_URL` armbian-next: u-boot: introduce `UBOOT_DEBUGGING=yes` which enables `CONFIG_LOG=y` and others armbian-next: atf: add `LOG_LEVEL=40` (up from default 20) and `BUILD_STRING="armbian"` to atf make invocation armbian-next: back to ANSI logs by default; HTML logs only by `EXPORT_HTML_LOG=yes` armbian-next: u-boot: extract into functions; add hooks - for more flexibility armbian-next: python: info: replace instances of ${SRC} with literal `${SRC}` so infos are more comparable armbian-next: `check_loop_device()` now uses `do_with_retries 5 ...` to give the OS time armbian-next: introduce `KERNEL_EXTRA_DIR` to suffix `LINUXSOURCEDIR` forcefully armbian-next: kernel-localmodconfig: update `kernel_config_mtime` after running `localmodconfig`, otherwise changes not detected armbian-next: traps: cleanups: run cleanups in the reverse order they were added; allow running one single cleanup and removing from list armbian-next: cleaning: fix shortcircuits, warn about unknown clean levels armbian-next: fix logging headers in HTML/ANSI log output armbian-next: WiP: u-boot: add fasthash to u-boot armbian-next: u-boot: add debug info when hacking `CONFIG_ENV_IS_IN_EXT4` in u-boot `.config` armbian-next: `write_image_to_device`: if `SKIP_VERIFY=no`, don't waste time checksumming the image, it won't be used armbian-next: logs: avoid errors by cwd'ing to SRC armbian-next: uboot: use CFLAGS/KCFLAGS everywhere; run make through `unbuffer` (from `expect` package) for full color logging armbian-next: kernel: run kernel's make through `unbuffer` as well armbian-next: cleaning: don't complain about 'none' `CLEAN_LEVEL` armbian-next: sources/families: meson64: gx: use `python2` explicitly for calling `acs_tool.pyc`; better logging armbian-next: prepare a bin dir inside WORKDIR with a `python` symlink to `/usr/bin/python2` and add it to `PATH` armbian-next: retry apt update & apt install family packages in addition to board packages armbian-next: fix for tests of `BOOTSCRIPT` and `BOOTCONFIG!=none` armbian-next: split `early_prepare_host_dependencies()` (run early, during config, before aggregation) from `prepare_host()` (run after aggregation) armbian-next: re-add dependency `fdisk` after juggling rebases armbian-next: introduce `ARMBIAN_LOGS_TO_JOURNAL` and `ARMBIAN_LOGS_JOURNAL_IDENTIFIER` (logs `display_alert` to journald) armbian-next: debugs about armbianEnv.txt overlays/fdtfile armbian-next: deploy bootscript even if BOOTCONFIG=none armbian-next: HACK: ROOTFSCACHE_VERSION is undefined and failing everytime armbian-next: fix "create list of installed packages for debug" armbian-next: many fixes after v29 rebase; mostly involving umount armbian-next: let all unmounting of `${MOUNT}` be done by `umount_chroot_recursive` armbian-next: less verbose umount_chroot(), except after first try, so we can know what is left mounted armbian-next: bye bye `$FAST_CREATE_IMAGE`: always use `truncate` for blank image armbian-next: normalize mount/umount `$target`s to have (or not) the trailing slash armbian-next: move `resolv.conf` symlink to systemd to post_debootstrap_tweaks so customize phase has working DNS armbian-next: grub: call `VER="generic" update_initramfs` when `"${DISTRO_GENERIC_KERNEL}" == "yes"` (so ddk builds work again) armbian-next: `umount_chroot_recursive`: don't try to `realpath` unless it's a dir to begin with armbian-next: don't break if parsing `BOOT_SOC` out of `BOOTCONFIG` fails; sanity checks for `BOOT_SOC`/`DDR_BLOB` armbian-next: export `CHOSEN_KERNEL_WITH_ARCH` again, for reporting only armbian-next: `kernel`: add hooks `fetch_sources_for_kernel_driver` and `patch_kernel_for_driver` to allow migrating EXTRAWIFI stuff to extensions armbian-next: logging: don't bomb due to lack of `git` or `zstdmt` armbian-next: traps: don't duplicate error message/stacktrace when error occurs in subshell; instead indicate clearly `SUBSHELL` armbian-next: show `mkfs` ext2/ext4 output (remove `-q`), it's being run under the logging manager already armbian-next: the great cli entrypoint (+docker) rewrite; introduce `USE_LOCAL_APT_DEB_CACHE` replacing `apt-cacher-ng` armbian-next: always include `libc6-amd64-cross` in hostdeps; `qemu` is a dummy package now, remove armbian-next: add `gcc-riscv64-linux-gnu` to hostdeps (`crossbuild-essential-riscv64` is Ubuntu-only) armbian-next: WiP/TODO: `extras/buildpkg.sh` code (eg: chroot_installpackages_local) has not been handled yet, warn and don't run it armbian-next: retry 3 times for loop device's size, then give up; introduce `RETRY_RUNS` in `do_with_retries()` armbian-next: back to including 'cli' `sources/apt` sources when building non-desktops (eg, nala etc) armbian-next: make sure we're running on Bash 5.x at least armbian-next: make sure we've coreutils armbian-next: don't force SHOW_LOG=yes during Dockerfile build; we're verbose enough without it armbian-next: docker: try support Rancher Desktop (in dockerd mode), experimental. armbian-next: docker: don't even try to to prepare Docker, if it's not available armbian-next: cleanup `extensions` logging, using `display_alert` new levels `extensions` (`SHOW_EXTENSIONS=yes`) and `extensionstrace` (`SHOW_EXTENSIONS_TRACE=yes`) armbian-next: cleanup logging in distro-agnostic.sh; don't use `install.log` or such armbian-next: `mountpoints.sh` with all bind/volume directories definitions and looping func; use it to determine best bind/volume combination for each host OS in Docker armbian-next: initial post-`kinetic` `resolved` debacle, for now just WARN don't break; missing resolved package in kinetic? armbian-next: `docker`: handle the case where we can't pull, and don't otherwise have, the base image; in this case use original debian/ubuntu image armbian-next: don't `uuidgen` twice for no reason armbian-next: `docker`: correctly show all Armbian Docker volumes; use bind, not volume, for `output/debs` armbian-next: correctly test for systemd `is-system-running --quiet` armbian-next: better handling for `binfmt_misc` woes; don't require it when native build & only complain if it really fails armbian-next: avoid git cramps when deciding to init repos armbian-next: show more complete gcc versions in logs armbian-next: `cli`: de-hardcode `build` and `docker`'s interdependence, via `DOCKER_CLI_CMD` armbian-next: `cli`: a bit more untangling of re-launching; introduce `ARMBIAN_CLI_RELAUNCH_COMMAND` and `cli_standard_relaunch_docker_or_sudo()` armbian-next: remove usage of `REQUIREMENTS_DEFS_ONLY`; initialize extensions for `requirements` cli command & docker pre-game deps armbian-next: rename `compile_plymouth-theme-armbian()` to `compile_plymouth_theme_armbian()` for consistency armbian-next: `json-info` command, wrapping up the Python tooling; running in Docker; fix config dump armbian-next: introduce hook `post_family_config_branch_<branch>()` as convenience after `post_family_config()` hook armbian-next: docker: locale-gen + enable all existing extensions before calculating docker hostdeps armbian-next: docker: disable showing Docker volume usage; fix syntax in test for `ARMBIAN_RUNNING_IN_CONTAINER` armbian-next: the great aggregation rewrite: pt 0: `SHOW_AGGREGATION` debug logging armbian-next: introduce `ARMBIAN_FIRMWARE_GIT_SOURCE` and `_BRANCH` to allow custom firmware repository coordinates armbian-next: git: do not delete working copies on remote origin mismatch; accept empty origins as valid armbian-next: u-boot: use `scripts/config` instead of `sed`ing into the `.config` (BOOTDELAY, CONFIG_LOGLEVEL=6, etc); v2022.10+ only armbian-next: `distcc`: optionally using `distcc` for kernel build armbian-next: `cli`: `distccd` launcher for build farm armbian-next: better `ccache` logs/stats/Docker-support/use in u-boot/kernel armbian-next: `call trace` analyzer and call-graph generator armbian-next: include `packages.external` and apt `sources` only in DESKTOP builds (pre-`aggregation.py`) armbian-next: `aggregation.py` rewrite; introduce `package-lists.sh`; usage in core lib armbian-next: `aggregation.py` and `package-lists.sh` usage in core grub-* extensions armbian-next: `grub`: fix/hack for grub deploy inside Docker/chroot when /dev/disk/by-uuid is not present; add sanity checks armbian-next: `vmware-vm` core extension: enables `image-output-ovf` and adds some cli and desktop packages to image list armbian-next: `image-output-utm` core extension: produces qcow2-based UTM ZIP image armbian-next: introduce `PREFER_DOCKER`, which acts the same as not-being-root: uses docker always armbian-next: `git`: add `origin` remote if it is empty armbian-next: REVERT: remove the BUILD_ONLY `build-tasks.sh` which makes no sense in armbian-next armbian-next: `qcow2`: introduce `QCOW2_KEEP_IMG=no`, which deletes the original .img after qcow2'ing it armbian-next: hostdeps: python2/3: add `setuptools` (required by some u-boot configs) armbian-next: `u-boot`: introduce hook `pre_config_uboot_target()` armbian-next: `qemu-uboot-arm64` and `qemu-uboot-x86` new boards armbian-next: fix for grub's `DISTRO_GENERIC_KERNEL` on Debian (not Ubuntu) during update-initramfs armbian-next: docker: implement `docker-purge` and `docker-shell` cli commands; deprecated mountpoints armbian-next: logging: helpers for handling special logging if stdout is a terminal (user-interactive) armbian-next: non-Docker cleaning of deprecated mountpoints (before general_cleaning) armbian-next: kernel: use `git worktree` & kernel.org clone bundles for kernel working copies armbian-next: u-boot: use `git worktree` for u-boot; full mainline clone for bare armbian-next: logging: add Markdown logging output armbian-next: git: `fetch_from_repo()`: determine, use for checkout, and set the SHA1 rev in `checked_out_revision` var armbian-next: introduce `INSTALL_HOST_TOOLS=no` to skip rkbin, fips, etc. for fast(er) development armbian-next: squash some `warn` loggings that were undue; remove `$SUDO_USER` handling in prepare-host.sh armbian-next: fix all `shellcheck` `errors` in `lib/` armbian-next: `shellcheck` downloader/runner; only for `lib/` armbian-next: disable `cpufrequtils` by default; to enable set `CPUFREQUTILS_ENABLE=true` [HIGHLY CONTENTIOUS CHANGE] armbian-next: fixes for interactive config; change wording of stuff and sort the list of boards in "expert" mode armbian-next: `docker`: fix when using `buildx`, always pass `--load` so produced image is actually available to Docker proper armbian-next: a few fixes (shortcircuits...) and enhancements for the GUI building; thanks NicoD armbian-next: introduce `SKIP_ARMBIAN_ROOTFS_CACHE` which is `=yes` by default armbian-next: introduce internal base tmp dir; check it early for sanity ref `noexec` and `nodev` mount options armbian-next: Python patching delusion, pt1 & pt2 & pt3 armbian-next: introduce ORAS-based kernel git bare tree seeding/bundles armbian-next: Python patching delusion, pt4: series & better markdown armbian-next: Python patching delusion pt 5 + EXTRAWIFI patch generator/harness + consistent patch mtime targets armbian-next: make sure at least one board config file is sourced, otherwise misspell `BOARD=` is hard to find armbian-next: `docker`: don't try to double init when recovering from failed pull armbian-next: bring `find_toolchain()` back from the dead; force `SKIP_EXTERNAL_TOOLCHAINS=no` to force use external toolchains armbian-next: fix `sunxi`/`sunxi64` using wrong KERNELPATCHDIR and incurring in old symlinks armbian-next: Revert "Fix issues 4589 (#4614)" (installing HEADERS dependencies manually) armbian-next: remove spurious `output.log` redirections in downloads.sh; `warn` that `download_and_verify()` is unhandled in armbian-next armbian-next: `git worktree`: forcefully fix the worktree's reference to the bare repo, and vice-versa armbian-next: ignore git errors due to missing 'origin' remote and/or revisions of `${SRC}/.git` during main-config armbian-next: git: setup the `worktree` first before trying to ensure safe armbian-next: drop old code for patching & fasthash armbian-next: better logging for `jethub`'s tweaks armbian-next: **regen library** after some 13 squashes (and repo.sh removal) armbian-next: aggregation: fix `PACKAGE_LIST_BOARD_REMOVE`/`PACKAGE_LIST_FAMILY_REMOVE`; also remove pkgs from DESKTOP aggregation armbian-next: `exec` into `sudo` when relaunching (AR-1452) armbian-next: validate cmdline params against malformed armbian-next: `reset_uid_owner` for the markdown log file too armbian-next: remove `@TODO` for stuff already done armbian-next: `git`: use `--recurse-submodules=no` for fetch; `fetch_from_repo()` handles submodules separately armbian-next: refactor `losetup -d ...` into `free_loop_device_insistent()` which retries 10 times armbian-next: `git`: mark submodule's dirs as safe before recursive `fetch_from_repo()` armbian-next: Python patching: pass `$HOME`/`$PATH` to Python, so git can find `.gitconfig` and Python can find `git` armbian-next: add (trap) cleanup handler to reset `./tmp` and `./output` to original pre-sudo UID before exiting, even with failure/abort armbian-next: `json-info`: do not replace ${SRC} armbian-next: fix: only aggregate packages if building images (AR-1471) armbian-next: only put `systemd-resolved`'s symlink in place if _not_ using NetworkManager armbian-next: Python tools (aggregation/patching): use separate pycache dir in ./cache; use unbuffered stdout/stderr armbian-next: tune logging in multiple places, there was a lot of redundant/un-needed info's armbian-next: better logging in `distro-agnostic.sh` armbian-next: remove very old extlinux killswitch armbian-next: happy new year, bump Igor's copyright armbian-next: `grub`: make `/usr/share/desktop-base/grub_background.sh` executable, otherwise does nothing armbian-next: `drivers-harness`: remove commit-date, do proper logging & escaping for git add/commit/format-patch armbian-next: `drivers-harness`: include the revert-IPX patch as a "driver" before all others armbian-next: `mvebu.conf::family_tweaks()`: small armbian-next'ify for better logging armbian-next: clearfog/cubox/odroidxu4: patching Makefile `armv5` to `armv7-a` so uboot compiles with modern gcc armbian-next: `rockchip-rk3588` `edge` bump & armbian-next'ify armbian-next: remove spurious debug logging with no useful info "Error-related files found" WiP: try for working armhf kernel-headers. leading nowhere. move to kernel-source / make modules_prepare later. WiP: kernel-headers via full source + `make modules_prepare` for 4.19 kernels. EXPERIMENTAL armbian-next: move compilation vars (chtreads, ccache stuff) from config to compile armbian-next: move var `HOSTRELEASE` from config to `prepare_host()` armbian-next: delete old aggregation code from config armbian-next: call hook `post_aggregate_packages()` only after actually aggregating armbian-next: avoid `linux-version` during configuration; `declare -g` instead of `export` armbian-next: config: reduce bash scope & remove some vars un-needed outside of config armbian-next: `cli`: `config-dump`: can now run on any bash/coreutils-enabled machine; remove some useless vars from dump armbian-next: `cli`: `json-info`: make errors clearer; don't include logs on success; small fixes armbian-next: `drivers-harness`: pass HOME/PATH explicitly to git commit/format-patch armbian-next: `drivers-harness`: make sure cached patch is valid; write it first to tmp file; redirect inside subshell armbian-next: split of `config-prepare` into multiple functions, allowing for logging of all non-interactive sections armbian-next: [focal-host] try to support `focal`-like host OS's again, via `python3.9` (and drop `buster` completely) armbian-next: [focal-host] try to support `focal`-like host OS's again, by avoiding git's `--no-auto-maintenance` armbian-next: [focal-host] remove usages of `declare -I`, not supported under focal's `bash` armbian-next: [focal-host] run `pip3` under the `python3_binary_path` (so coerce it to use 3.9 under focal) armbian-next: [focal-host] don't use git format-patch's `--no-encode-email-headers` which is not supported on focal's git armbian-next: [focal-host] don't use git's `--initial-branch` which is not supported on focal's git armbian-next: move aggregation to after prepare_host(), and thus out of config, and with correct Python deps armbian-next: patching: to-git: don't fail if is_autogen_dir; only change Subject if not rewriting in place armbian-next: patching: add sanity checks for magic markers matching mbox len & patch_contents not containing magic markers armbian-next: patching: go through parsing & splitting all patches before stopping if there are critical errors armbian-next: copy modified config + defconfig back from where it came from when `KERNEL_CONFIGURE=yes` armbian-next: uefi-common: `declare -g`, not `export` armbian-next: patching: don't gag if empty hexsha / no archeology possible armbian-next: patching: patches-to-git: don't eat newline between subject and message armbian-next: patching: grouping tryout, very basic by-dir: `SPLIT_PATCHES=yes` armbian-next: patching: use `--zero-commit` instead of find/replace for rewrite armbian-next: further split of `config-prepare()` and `do_main_configuration()` into `do_extra_configuration()` - fixes interactive desktop armbian-next: `SYNC_CLOCK`: don't attempt if `ntpd` is running; don't fail if `ntpdate` fails for any reason armbian-next: rootfs: don't stop if `df -h` fails armbian-next: `drivers-harness`: force `-c commit.gpgsign=false` during `git commit` Docs: slides Docs: slides (updated 28/Dec/2022) Docs: slides (updated 11/Jan/2023) armbian-next: `CLEAN_LEVEL=make-kernel` now does `git clean -xfd` instead, faster and 100% clean armbian-next: better logging for early apt installs during `prepare_host_basic()` armbian-oleg: handle error during host deps installation (Oleg has a mangled sources.list?) armbian-oleg: introduce `DOCKER_SIMULATE_CLEAN=yes` so I can pretend to be Oleg, but using Docker armbian-oleg: split hostdeps again, full of ifs, and reduce the minimum set of pkgs for Oleg-conditions while trying to keep Docker full armbian-next: `apt-cacher-ng` is now optional and activated via `MANAGE_ACNG=yes`; drop `NO_APT_CACHER` armbian-next: remove old template Dockerfile, -next does not use this at all armbian-oleg: `junk`: drastically reduce host-side dependencies / "remove junk" armbian-next: curb down wget logging for ORAS tooling download armbian-next: curb down `git fetch` logging; be verbose if user on terminal & not logging armbian-next: curb `aria2c`'s complaining a bit armbian-next: curb `free_loop_device_retried()`'s logging on first try armbian-next: long overdue split of `logging.sh` armbian-oleg: logging: new logfile format; `SHOW_LOG=yes` default; introduce `DEBUG=yes` armbian-next: fix quoting of retry var (cosmetic) armbian-oleg: better logs; only include non-empty .log files armbian-oleg: curb `apt-get`'s logging with `-q` / `-qq` armbian-oleg: curb Python launcher logs armbian-oleg: curb `rsync` logging during bsp armbian-oleg: drastically curb `update-initramfs` logging, unless SHOW_DEBUG=yes armbian-oleg: drastically curb `boot_logo()` logging armbian-oleg: drastically curb `apt-get` logging when running in chroot armbian-oleg: make `chroot_sdcard_apt_get_install_dry_run()` logging more useful by filtering the noise out armbian-oleg: if commands fail, write to log in bright red armbian-next: split `compilation/debs.sh`; `compile_xilinx_bootgen()` moved to family armbian-oleg: curb logging from building `armbian-firmware` armbian-oleg: curb logging from building `armbian-plymouth-theme`; do it in a logging section, like the others armbian-next: logging: reinit logging after processing cmdline parameters armbian-next: cli: half-assed, legacy based, `kernel` and `u-boot` cli commands armbian-next: `odroidxu4` vs kernel: move firmware hack to family using extension hook armbian-oleg: logging: drastically curb / make more useful kernel packaging logging armbian-oleg: logging: disable debugging (set -x) of generated postinst/etc kernel scripts (board-side, but also chroot logging) armbian-next: kernel: deterministic `.config` mtime handling; kernel build logging sections reorg armbian-next: remove `REPO_STORAGE` and `REPO_CONFIG` -- unused armbian-next: consider "Provided" installed packages when determining what is missing from host-side dependencies armbian-next: completely remove `acl` (which provided `getfacl`) and it's usages (basic deps) armbian-next: don't copy the host's `keyrings` to image armbian-next: Python tooling: use consolidated+hashed+cached pip base/pycache; don't pip during Dockerfile build, nor cli-requirements armbian-next: add core extension `c-plus-plus-compiler` which hostdeps on `g++-aarch64-linux-gnu` and enable it in JetHub family armbian-next: core extensions for `xfs` / `f2fs` / `brtfs` / `LUKS/cryptroot` hostdeps; enable in main-config armbian-next: Docker: abstract away and only run "docker info" once, it's very expensive armbian-next: `rootfs`: bunch'o'fixes, introduce `disable_systemd_service_sdcard()` to stop repeating incantantion armbian-next: don't warn about using extlinux armbian-next: a bunch of checks for Darwin; use brew's GNU coreutils first in PATH, so we get the correct utils; give instructions armbian-next: better logging for `kernel_prepare_bare_repo_from_oras_gitball()`; remove dead code armbian-next: fix re-launching of CLI commands under sudo (Docker was ok) armbian-next: `patching`: don't complain about missing files when the patch parsed OK and file was actually deleted armbian-next: `patching`: don't rewrite empty desc as "None" armbian-next: `patching`: remove quotes from author name during parsing armbian-next: `patching`: avoid mangling valid utf-8 from mbox'es armbian-next: `patching`: parse renamed source files, don't swallow them during rewrite armbian-next: optimize `config_source_board_file()`, now 600x faster by not reusing (slow) code from interactive armbian-next: optimize for `CONFIG_DEFS_ONLY=yes`, skipping stacktraces/git info armbian-next: introduce `POOR_MAN_PROFILER=yes` (under `ANSI_COLOR=none`) armbian-next: `docker`: fix Dockerfile output indentation & warning msgs about generation armbian-next: make sure temporary `kernel` and `u-boot` CLI commands always build a kernel/u-boot even if cached armbian-next: debug note about fstab with tmpfs in chroot armbian-next: mark Vagrant CLI as unimplemented armbian-next: `docker`: use dash-lines before/after launching docker; set `ARMBIAN_INSIDE_DOCKERFILE_BUILD=yes` when it is so armbian-next: `traps`: allow for cleanup handlers with arguments; unify around `run_one_cleanup_handler()` armbian-next: `ccache`: show more in ccache debugging armbian-next: make superglobals readonly (`DEST/WORKDIR/LOGDIR/SDCARD/MOUNT` and others) armbian-next: introduce `tmpfs-utils.sh`; put `LOGDIR` and `WORKDIR` under tmpfs; set `CCACHE_TEMPDIR` under `WORKDIR` armbian-next: split `prepare_host()`, fix `.tmp` reset trap armbian-next: do basic host checks before config if interactive, or during prepare-host if not armbian-next: tmpfs-utils: last-resort debugging, to stderr, of failed tmpfs unmount armbian-next: logging: squash nasty leaking file descriptor bug due to "tee" armbian-next: bring back `swig` hostdep -- needed for some u-boot's armbian-next: severe bug, countdown counted up armbian-next: kernel: cleanup bundle after patching succeeded, not after build success armbian-next: firmware: don't build `-full` firmware if not on CI/noninteractive and board's not going to use it armbian-next: `chroot_sdcard_apt_get_update()` to replace and better log `chroot_sdcard_apt_get update` armbian-next: extensions: rename `kernel-localmodconfig` to just `lsmod` armbian-next: docker: remove old code armbian-next: try harder to get `en_US.UTF-8` logging armbian-next: drop dead/duplicated code for bootsplash and mark kernel-drivers.sh (which includes bootsplash) as dead code armbian-next: extract `move_images_to_final_destination()` and optimize for fast-move when src/dst on same filesystem armbian-next: tune logging: unmounting, u-boot prep/source/install; traps; extensions armbian-next: don't leak `.raw` image if failure before move; show rootfs-tmpfs info armbian-next: `mktemp` and `tmpfs` related utility functions with automatic cleanup handlers armbian-next: small cleanups: squash typos / add function keyword / add types / mark dead code armbian-next: small cleanups: squash typos / add function keyword / add types / mark dead code armbian-next: add logging with size of actual built rootfs in MiB; debug tmpfs in trap & after pkgs done armbian-next: still fighting `tee` leaking under duress, and I think I won armbian-next: `compile_firmware()`: use new temp dir helpers (saves 2Gb+ in WORKDIR), fix typos armbian-next: severe bug, there was no prefix-emoji for Windows armbian-next: allow shortcut `EXT=` to the overly-long and plural `ENABLE_EXTENSIONS=` armbian-next: use retries for downloading ORAS tooling armbian-next: aggregation.py vs create-cache.sh: better hashing, much desktop armbian-next: patching.py vs decent logging: better, but not there yet Docs: slides (not-updated 18/Jan/2022) armbian-next: nextify "Re-enabling support for Rpi3" armbian-next: still fighting `tee` leaking under duress, turns out I _hadn't_ really won armbian-next: still fighting `tee` leaking: now I think I won, again armbian-next: somewhat-better handling/logging when using Docker (don't show two logfiles to inspect) armbian-next: rename too-long main function to `prep_conf_main_build_single()` armbian-next: hopefully improved error logging; avoid logging errors up to 3x; subdued red stacktraces armbian-next: `patching`: less horrible patching log, pt 37 armbian-next: fix missing `BOARD_TYPE` armbian-next: `git`: show "fetch completed" info, lest user thinks fetch is hanging while checking out armbian-next: `patching`: less horrible patching log, pt 38: downgrade utf-8 mbox to log warning, mark as problem armbian-next: fix ATF build under binutils 2.39+ via `TF_LDFLAGS=--no-warn-rwx-segment` armbian-next: add Debian Ports keyring hostdep to strap Debian riscv64 from ports armbian-next: `countdown`: split off file for countdown functions (countdown-to-abort and countdown-to-continue) armbian-next: `kernel`: `git`: split via-ORAS and via-Bundle tangle armbian-next: `kernel`: `git`: remove dead code, `GIT_PRE_FETCH_HOOK` / `checked_out_revision_mtime` armbian-next: replace bare `sync` invocations by introducing `wait_for_disk_sync()`; sync early at start of build armbian-next: adaptive shallow/full Kernel git bare git (3gb -> 300mb download) armbian-next: fix, don't try to get pid descendants on non-Linux (fixes Darwin cleanups) armbian-next: darwin: try to get rid of `.DS_Store` sneaky bastards before launching Docker armbian-next: back to cleaning downloaded bundles/gitballs after they've converted to bare tree & proven working armbian-next: git: detect, and stop, when git bare trees vs worktrees have been shuffled around and can't be recovered armbian-next: git: disallow forcing impossible shallow/full; warn if it's the case armbian-next: docker: archive logfiles again; warn & greet testers when archiving many old logfiles armbian-next: logging: new log file / summary nomenclature armbian-next: bsp: use `prepare_temp_dir_in_workdir_and_schedule_cleanup()`; bring back `PACKAGES_SEARCH_ROOT_ABSOLUTE_DIRS` armbian-next: firmware-deb.sh: prefix temp dir so it's "deb-firmware" armbian-next: extensions: move 'extensions.sh' into library; replace manual source with init function armbian-next: fix ANSI logs, use reset at strategic places armbian-next: `docker`: don't bomb if stdin is not a terminal (eg, in CI) armbian-next: `docker`: pass `CI` env down to Docker (for GHA etc) armbian-next: python/pip: solve crazy with `/usr/bin/pip3` by not using it (instead, `python3 -m pip`); don't mask errors armbian-next: `docker`: warn & countdown if we can't write to `cache/docker`; continue by using `.tmp/docker` if not aborted. armbian-next: countdown: if to test `-t 0` (0: stdin, 1:stdout) for interactive-ness; also test in if-not-aborted version armbian-next: interactive: info msg before (slow) gathering of all boards armbian-next: logging: (very, very early beta) for uploading/sharing of ANSI log file online armbian-next: kernel/logging: stop lying about LD/AR/INSTALL/SIGN/XZ make output armbian-next: patching: accept b4's magic marker `git@z Thu Jan 1 00:00:00 1970` for mbox validations armbian-next: mark `remove_all_trap_handlers()` deprecated, it never did anything armbian-next: downloads.sh: some cleanup and add debugging to `get_urls()` armbian-next: `logging`: validate where not already in a logging section when trying to start a new one armbian-next: `apt`: expand local caching to include package lists, as well as .deb's armbian-next: `main`: split `default-build` into `build-packages`, `start-end` armbian-next: `rootfs`: split `create-cache` into `rootfs-create` armbian-next: `rootfs`: new CLI command for rootfs cache building; further refactorings armbian-next: `rootfs`: adapt to new reality after huge refactor armbian-next: `aggregation`: dont' require or consider `BOARD` or `LINUXFAMILY` for aggregation anymore armbian-next: `rootfs`: build rootfs from CLI, sans-BOARD, sans-FAMILY, etc -- rootfs are meant to be shared armbian-next: `rootfs`: reset UID of produced rootfs tarball; announce what is doing and what has done armbian-next: `rootfs`: converge `rootfs-cache` and `rootfs` together; exchange BOARD for ARCH armbian-next: use unified methods for tmp dir handling all around (except ATF, which is special) armbian-next: extensions: `nomod`: for when you really don't care about modules armbian-next: we don't really need target `RELEASE` defined to install host deps (for now), don't freak out about it armbian-next: kernel-debs: re-enable postinst debugging; fix bad builddeb port for zImage armhf (image_name == NAME_KERNEL) armbian-next: cleanup, kill and add to-do's, remove dead code, turn down logging, squash future shortcircuit bugs armbian-next: extrawifi: include `driver_uwe5622_allwinner()` in harness armbian-next: extrawifi: include all `patch/misc/wireless*.patch` and `patch/misc/*wifi*.patch` in drivers hash calculation armbian-next: kernel-debs: force `arch/arm/boot/zImage` as vmlinuz for armhf armbian-next: kernel-debs: squash some warnings; add `KERNEL_SRC_ARCH` to arch configs; don't warn about `NAME_KERNEL` armbian-next: wait_for_disk_sync(): warn only if more than 30s fsync armbian-next: binfmt: make sure we have all `binfmt`'s setup & enabled for `rootfs` build (v2) armbian-next: `docker`: don't spew errors about sharing cache if cache doesn't even exist armbian-next: GHA & rootfs adventures, pt 1 armbian-next: aggregation: quick & dirty "potential paths" in Markdown armbian-next: cli: `kernel` cli forces `KERNEL_CONFIGURE=no`; @TODO this should only "default to", not force. for proper kernel cli, later armbian-next: logging: pure ASCII .log; included in Markdown for GHA armbian-next: docker/gha: pass down a bunch of GHA specific vars (list unconfirmed) armbian-next: rootfs: curb logging; GHA outputs `rootfs_cache_id_version`, `rootfs_cache_id` and `rootfs_out_filename_relative` armbian-next: ext: `grub-riscv64`: syncs with original version, fixes, nextify armbian-next: ext: `grub-sbc-media`: syncs with original version, fixes, nextify Docs: slides (1-slide update 25/Jan/2023) armbian-next: drop buildpkg code armbian-next: patching: rewrite: don't gag on bad "/dev/null.c" rename patches; archeology: `--find-copies-harder`; try to get GHA to run GH pages armbian-next: chroot_sdcard_apt_get_install_dry_run(): run again if first try fails, with more logging armbian-next: `docker`: passthrough of the SSH agent, via Docker magic on Darwin; enable for `docker-shell` and `kernel-patches-to-git` armbian-next: `docker`: include `${SRC}/.git` static via Dockerfile, for `kernel-patches-to-git` / archeology; use fast archeology armbian-next: all chroots: mount `/var/tmp` as tmpfs; create dir `/run/user/0` and mount as tmpfs @TODO cleanups armbian-next: docker: pass `ARMBIAN_ENABLE_CALL_TRACING=yes` via env, and only if `DOCKER_ARMBIAN_ENABLE_CALL_TRACING=yes` armbian-next: compress-checksum: move config to config-prepare; drop weight; kill warnings armbian-next: repeat build opts: fix missing BOARD, show both after config (for logfile) & after build armbian-next: bump logging archive greet to 14 files armbian-next: fix: kernel-git-oras: make sure `cache_git_bare_dir` exists before trying to decide which media it's on armbian-next: git: `fetch_from_repo()`: add fetched_revision(_ts) as well as checked_out_revision(_ts); introduce param `do_checkout=no` to only fetch, not checkout armbian-next: cli-build: don't show two `BOARD=` repeat option armbian-next: aggregate_packages out of `main_default_start_build()`; introduce `assert_requires_aggregation()` armbian-next: firmware-deb: reworked: Provides/Conflicts instead of Replaces; `cp` -> `git archive`; new firmware CLI; FULL version .deb out of tmpfs armbian-next: unify 2 methods for config: `prep_conf_main_minimal_ni()` (rootfs/firmware/kernel) and `prep_conf_main_build_single()` (full-build, possibly interactive) armbian-next: curb some logging for git non-checkouts armbian-next: prepare-host: don't manage ACNG in prepare if not asked to armbian-next: docker: fix: not all `find`'s are made equal, back to `-mtime +1` (full 24-hour period needed for match) armbian-next: interactive: review `-t 0` instead of `-t 1`; move non-interactive `check_basic_host()` from prepare-host to config-prepare, for consistency armbian-next: move aggregation-had-run check to more strategic places (pre-bsp, pre-rootfs) instead of at the beginning of full_build_packages_rootfs_and_image() armbian-next: docker: split `docker_cli_build_dockerfile()` into `docker_cli_prepare_dockerfile()`; introduce `FAST_DOCKER` armbian-next: git: refactor `git_parse_ref()` out of `fetch_from_repo()` for reuse armbian-next: config: twist `prep_conf_main_minimal_ni()` to receive `use_board=` armbian-next: do-not-fix docker-asking-for-docker error by adding a "tip" armbian-next: cli: *do not* allow running as root on terminal; make `PREFER_DOCKER=no` actually work armbian-next: cli: try (and fail, probably...) to be helpful with half-working Docker installs WiP: artifacts: kernel - pt1: versioning WiP: artifacts: kernel - pt2: memoizing git ref2info WiP: artifacts: kernel - pt3: drivers+patches+.config hashing WiP: artifacts: kernel - pt4: squash unrelated bugs that show up; move `prepare_compilation_vars()` to default build WiP: artifacts: kernel - pt5: tune kernel version, refactor WiP: artifacts: u-boot - pt6: add artifact for u-boot WiP: artifacts: u-boot/kernel - pt7 - adapt legacy/artifact versions; use common `capture_rename_legacy_debs_into_artifacts()` armbian-next: logging: detect `TERM=alacritty` and use normal instead of gray armbian-next: fix: for the 1000th time had broken `KERNEL_ONLY=yes`; curb some warnings armbian-next: introduce `assert_prepared_host()` and assert before using any Python, to un-mask any further fallacies armbian-next: take a breath: config might mark aggregation required; aggregation requires prepare_host, and thus runs in default-build armbian-next: logging: ansi-haste sharing beta 2 (now with URL) armbian-next: docker: fix Dockerfile generation; don't output logs info during Dockerfile build armbian-next: u-boot: downgrade more errors to warnings, so we can build some legacy u-boot's armbian-next: cli: u-boot alias: don't stop to interactively ask for kernel configuration during u-boot build armbian-next: call-stack-analyze.py: detect missing function_to_file and stop properly (half-finished traces?) armbian-next: json-info: cleanup info/json/csv/opensearch, add logging, add some very basic instructions; add Kibana dashboards & docker-compose to bring it OS+Kibana up armbian-next: shellcheck: update, cleanup script, invent some new severities armbian-next: shellcheck: squash a bunch of 'critical' warnings armbian-next: shellcheck: raise the bar, stop excluding "SC2155 Declare and assign separately to avoid masking return values" armbian-next: squash some shellcheck warnings; don't `local` and assign subshell value in same statement, that masks errors (`declare`/`local` almost-always works) armbian-next: shellcheck: raise the bar, stop excluding "SC2154 is referenced but not assigned" armbian-next: shellcheck: squash more shellcheck warnings; some long-lost variables being used; some unset's that are better reset's, etc armbian-next: `docker-shell`: provide SSH and forward ssh auth socket to `docker-shell` armbian-next: docker: pass `CARD_DEVICE` to Docker via `--device` armbian-next: u-boot: mark bare repo as safe before using it (@TODO `git_ensure_safe_directory()` stuff, again) armbian-next: swap github in as default u-boot mirror; allow `denx` to use upstream's gitlab armbian-next: git ref2info: dereference annotated tags to get "real" commit SHA1; fail if curl failed; validate parse_makefile_version() armbian-next: hash/cache: remove some cruft, some better logging armbian-next: docker: pass down git info via env var, since Docker doesn't get `${SRC}/.git`, yet we need that info in the Docker logs armbian-next: u-boot: fix: check gcc version before trying to downgrade certain errors to warnings [kinetic] un-symlink `cli`/`main` `kinetic`->`jammy` packages; add `systemd-resolved` which was split off from main `systemd` package in Kinetic `risc-v`/`starfive`: new `starfive` family with their (vendor) kernel `risc-v`/`starfive`: use mainline kernel 6.1.y, with StarFive's rebased patches against `v6.1.5` `risc-v`/`starfive`: `CONFIG_MOTORCOMM_PHY=m` for the onboard Ethernet `odroidc2`: u-boot: use minimal patchset for v22.01 u-boot `meson64`: edge/6.2: spi-nor: add support for XT25Q64 in addition to XT25F128B `meson64`: edge/6.2: bump to `6.2-rc5` `meson64`: edge/6.2: rebase onto `v6.2-rc5` `odroidm1`: enable working userland `fw_setenv`, so users can go back to Petitboot & otherwise tweak u-boot env from Armbian `odroidm1`: bump to 6.2-rc3; look ma, no patches. `media`: fixes for bad mbox/unneeded patches `media-edge` and `media-current` - thanks to @schwar3kat `sunxi/sunxi64`: disable some sunxi/sunxi64 "readme" patches that contain other patches -- for discussion `rk322x-box`: fix relative path reference to .bin in patch (a-n changed u-boot path on disk) - for @paolosabatino `rockchip64`: current/5.15: remove IPX revert patch that's already in drivers now `rockchip64`: current/5.15: fix bad unidiff/lack of mbox of wifi-hf-lps170-bl602.patch & makefile odroidm1: edge: bump to `6.2-rc5` `sun50iw9`: half-fixes to get going under armbian-next (@TODO needs review/nextify/confirm busybox is not needed) WiP: bump edge to `v6.2-rc6` for UEFI / meson64 / rk3568-odroid / rockchip-rk3588 `meson64`: edge/6.2: arm64: dts: amlogic: meson-sm1-odroid-hc4: fix active fan thermal trip WiP: `odroidhc4`: u-boot to 23.01; add HC4-SPI target preboot for USB; change HC4-SD target defconfig to new `hc4_sd`, similar to `c4` Revert "Add some Wireless modules to RK3588 legacy kernel (#4759)" Docs: slides (2-slide update 01/Feb/2023) Igor <igor@armbian.com> (2): armbian-next: cli: add CLI alias command `rootfs-cache` for creating rootfs cache armbian-next: rootfs: introduce `ROOTFS_COMPRESSION_RATIO`, default 5 for general use, default 15 for `rootfs` CLI; fix pv | tar bug; remove GPG signing Lane Jennison <lane@lane-fu.com> (1): armbian-next: fix old fuction name refrence cleaning to general_cleaning Richard Neese <r.neese@gmail.com> (1): `risc-v`/`starfive`: switch from `grub` to `extlinux` Tony <tonymckahan@gmail.com> (1): armbian-next: officially support WSL2; pester user for UTF-8 terminal amazingfate <liujianfeng1994@gmail.com> (1): `rockchip64`: update bl31 and ddr blobs of rk3568 [cherry-pick]
-
git shortlog since last week...
Ricardo Pardini <ricardo@pardini.net> (74): armbian-next: nextify "Re-enabling support for Rpi3" armbian-next: still fighting `tee` leaking under duress, turns out I _hadn't_ really won armbian-next: still fighting `tee` leaking: now I think I won, again armbian-next: somewhat-better handling/logging when using Docker (don't show two logfiles to inspect) armbian-next: rename too-long main function to `prep_conf_main_build_single()` armbian-next: hopefully improved error logging; avoid logging errors up to 3x; subdued red stacktraces armbian-next: `patching`: less horrible patching log, pt 37 armbian-next: fix missing `BOARD_TYPE` armbian-next: `git`: show "fetch completed" info, lest user thinks fetch is hanging while checking out armbian-next: `patching`: less horrible patching log, pt 38: downgrade utf-8 mbox to log warning, mark as problem armbian-next: fix ATF build under binutils 2.39+ via `TF_LDFLAGS=--no-warn-rwx-segment` `media`: fixes for bad mbox/unneeded patches `media-edge` and `media-current` - thanks to @schwar3kat `rk322x-box`: fix relative path reference to .bin in patch (a-n changed u-boot path on disk) - for @paolosabatino armbian-next: `countdown`: split off file for countdown functions (countdown-to-abort and countdown-to-continue) armbian-next: `kernel`: `git`: split via-ORAS and via-Bundle tangle armbian-next: `kernel`: `git`: remove dead code, `GIT_PRE_FETCH_HOOK` / `checked_out_revision_mtime` armbian-next: replace bare `sync` invocations by introducing `wait_for_disk_sync()`; sync early at start of build WiP: adaptive shallow/full Kernel git bare git (3gb -> 300mb download) armbian-next: fix, don't try to get pid descendants on non-Linux (fixes Darwin cleanups) armbian-next: darwin: try to get rid of `.DS_Store` sneaky bastards before launching Docker armbian-next: back to cleaning downloaded bundles/gitballs after they've converted to bare tree & proven working armbian-next: git: detect, and stop, when git bare trees vs worktrees have been shuffled around and can't be recovered armbian-next: git: disallow forcing impossible shallow/full; warn if it's the case armbian-next: docker: archive logfiles again; warn & greet testers when archiving many old logfiles armbian-next: logging: new log file / summary nomenclature armbian-next: bsp: use `prepare_temp_dir_in_workdir_and_schedule_cleanup()`; bring back `PACKAGES_SEARCH_ROOT_ABSOLUTE_DIRS` armbian-next: firmware-deb.sh: prefix temp dir so it's "deb-firmware" armbian-next: extensions: move 'extensions.sh' into library; replace manual source with init function armbian-next: fix ANSI logs, use reset at strategic places armbian-next: `docker`: don't bomb if stdin is not a terminal (eg, in CI) armbian-next: `docker`: pass `CI` env down to Docker (for GHA etc) armbian-next: python/pip: solve crazy with `/usr/bin/pip3` by not using it (instead, `python3 -m pip`); don't mask errors armbian-next: `docker`: warn & countdown if we can't write to `cache/docker`; continue by using `.tmp/docker` if not aborted. armbian-next: countdown: if to test `-t 0` (0: stdin, 1:stdout) for interactive-ness; also test in if-not-aborted version armbian-next: interactive: info msg before (slow) gathering of all boards armbian-next: logging: (very, very early beta) for uploading/sharing of ANSI log file online armbian-next: kernel/logging: stop lying about LD/AR/INSTALL/SIGN/XZ make output armbian-next: patching: accept b4's magic marker `git@z Thu Jan 1 00:00:00 1970` for mbox validations armbian-next: mark `remove_all_trap_handlers()` deprecated, it never did anything armbian-next: downloads.sh: some cleanup and add debugging to `get_urls()` armbian-next: `logging`: validate where not already in a logging section when trying to start a new one armbian-next: `apt`: expand local caching to include package lists, as well as .deb's armbian-next: `main`: split `default-build` into `build-packages`, `start-end` armbian-next: `rootfs`: split `create-cache` into `rootfs-create` armbian-next: `rootfs`: new CLI command for rootfs cache building; further refactorings armbian-next: `rootfs`: adapt to new reality after huge refactor armbian-next: `aggregation`: dont' require or consider `BOARD` or `LINUXFAMILY` for aggregation anymore armbian-next: `rootfs`: build rootfs from CLI, sans-BOARD, sans-FAMILY, etc -- rootfs are meant to be shared armbian-next: `rootfs`: reset UID of produced rootfs tarball; announce what is doing and what has done armbian-next: `rootfs`: converge `rootfs-cache` and `rootfs` together; exchange BOARD for ARCH armbian-next: use unified methods for tmp dir handling all around (except ATF, which is special) armbian-next: extensions: `nomod`: for when you really don't care about modules armbian-next: we don't really need target `RELEASE` defined to install host deps (for now), don't freak out about it armbian-next: kernel-debs: re-enable postinst debugging; fix bad builddeb port for zImage armhf (image_name == NAME_KERNEL) armbian-next: cleanup, kill and add to-do's, remove dead code, turn down logging, squash future shortcircuit bugs armbian-next: extrawifi: include `driver_uwe5622_allwinner()` in harness armbian-next: extrawifi: include all `patch/misc/wireless*.patch` and `patch/misc/*wifi*.patch` in drivers hash calculation armbian-next: kernel-debs: force `arch/arm/boot/zImage` as vmlinuz for armhf armbian-next: kernel-debs: squash some warnings; add `KERNEL_SRC_ARCH` to arch configs; don't warn about `NAME_KERNEL` armbian-next: wait_for_disk_sync(): warn only if more than 30s fsync armbian-next: binfmt: make sure we have all `binfmt`'s setup & enabled for `rootfs` build (v2) armbian-next: `docker`: don't spew errors about sharing cache if cache doesn't even exist armbian-next: GHA & rootfs adventures, pt 1 armbian-next: aggregation: quick & dirty "potential paths" in Markdown armbian-next: riscv64: move `debian-ports-archive-keyring` from main to debootstrap; also only for `sid` meson64: edge/6.2: spi-nor: add support for XT25Q64 in addition to XT25F128B meson64: edge/6.2: bump to `6.2-rc5` meson64: edge/6.2: rebase onto `v6.2-rc5` armbian-next: cli: `kernel` cli forces `KERNEL_CONFIGURE=no`; @TODO this should only "default to", not force. for proper kernel cli, later armbian-next: logging: pure ASCII .log; included in Markdown for GHA armbian-next: docker/gha: pass down a bunch of GHA specific vars (list unconfirmed) armbian-next: rootfs: curb logging; GHA outputs `rootfs_cache_id_version`, `rootfs_cache_id` and `rootfs_out_filename_relative` Docs: slides (1-slide update 25/Jan/2022) ------------------------ armbian-next END marker ----------------------------- Igor <igor@armbian.com> (3): Add CLI alias command `rootfs-cache` for creating rootfs cache armbian-next: rootfs: introduce `ROOTFS_COMPRESSION_RATIO`, default 5 for general use, default 15 for `rootfs` CLI; fix pv | tar bug; remove GPG signing Clean and unify XFCE and Cinnamon (squashed)
-
Since I've no slides, a terrible sense of humor, and nothing else to post, here's a git shortlog of the changes since last week.
Ricardo Pardini <ricardo@pardini.net> (99): armbian-next: `CLEAN_LEVEL=make-kernel` now does `git clean -xfd` instead, faster and 100% clean armbian-next: better logging for early apt installs during `prepare_host_basic()` armbian-oleg: handle error during host deps installation (Oleg has a mangled sources.list?) armbian-oleg: introduce `DOCKER_SIMULATE_CLEAN=yes` so I can pretend to be Oleg, but using Docker armbian-oleg: split hostdeps again, full of ifs, and reduce the minimum set of pkgs for Oleg-conditions while trying to keep Docker full armbian-next: `apt-cacher-ng` is now optional and activated via `MANAGE_ACNG=yes`; drop `NO_APT_CACHER` armbian-next: remove old template Dockerfile, -next does not use this at all armbian-oleg: `junk`: drastically reduce host-side dependencies / "remove junk" armbian-next: curb down wget logging for ORAS tooling download armbian-next: curb down `git fetch` logging; be verbose if user on terminal & not logging armbian-next: curb `aria2c`'s complaining a bit armbian-next: curb `free_loop_device_retried()`'s logging on first try armbian-next: long overdue split of `logging.sh` armbian-oleg: logging: new logfile format; `SHOW_LOG=yes` default; introduce `DEBUG=yes` armbian-next: fix quoting of retry var (cosmetic) armbian-oleg: better logs; only include non-empty .log files armbian-oleg: curb `apt-get`'s logging with `-q` / `-qq` armbian-oleg: curb Python launcher logs armbian-oleg: curb `rsync` logging during bsp armbian-oleg: drastically curb `update-initramfs` logging, unless SHOW_DEBUG=yes armbian-oleg: drastically curb `boot_logo()` logging armbian-oleg: drastically curb `apt-get` logging when running in chroot armbian-oleg: make `chroot_sdcard_apt_get_install_dry_run()` logging more useful by filtering the noise out armbian-oleg: if commands fail, write to log in bright red armbian-next: split `compilation/debs.sh`; `compile_xilinx_bootgen()` moved to family armbian-oleg: curb logging from building `armbian-firmware` armbian-oleg: curb logging from building `armbian-plymouth-theme`; do it in a logging section, like the others armbian-next: logging: reinit logging after processing cmdline parameters armbian-next: cli: half-assed, legacy based, `kernel` and `u-boot` cli commands armbian-next: `odroidxu4` vs kernel: move firmware hack to family using extension hook armbian-oleg: logging: drastically curb / make more useful kernel packaging logging armbian-oleg: logging: disable debugging (set -x) of generated postinst/etc kernel scripts (board-side, but also chroot logging) armbian-next: kernel: deterministic `.config` mtime handling; kernel build logging sections reorg armbian-next: remove `REPO_STORAGE` and `REPO_CONFIG` -- unused armbian-next: consider "Provided" installed packages when determining what is missing from host-side dependencies armbian-next: completely remove `acl` (which provided `getfacl`) and it's usages (basic deps) armbian-next: don't copy the host's `keyrings` to image armbian-next: Python tooling: use consolidated+hashed+cached pip base/pycache; don't pip during Dockerfile build, nor cli-requirements armbian-next: add core extension `c-plus-plus-compiler` which hostdeps on `g++-aarch64-linux-gnu` and enable it in JetHub family armbian-next: core extensions for `xfs` / `f2fs` / `brtfs` / `LUKS/cryptroot` hostdeps; enable in main-config armbian-next: Docker: abstract away and only run "docker info" once, it's very expensive armbian-next: `rootfs`: bunch'o'fixes, introduce `disable_systemd_service_sdcard()` to stop repeating incantantion armbian-next: don't warn about using extlinux armbian-next: a bunch of checks for Darwin; use brew's GNU coreutils first in PATH, so we get the correct utils; give instructions armbian-next: better logging for `kernel_prepare_bare_repo_from_oras_gitball()`; remove dead code armbian-next: fix re-launching of CLI commands under sudo (Docker was ok) armbian-next: `patching`: don't complain about missing files when the patch parsed OK and file was actually deleted armbian-next: `patching`: don't rewrite empty desc as "None" armbian-next: `patching`: remove quotes from author name during parsing armbian-next: `patching`: avoid mangling valid utf-8 from mbox'es armbian-next: `patching`: parse renamed source files, don't swallow them during rewrite armbian-next: optimize `config_source_board_file()`, now 600x faster by not reusing (slow) code from interactive armbian-next: optimize for `CONFIG_DEFS_ONLY=yes`, skipping stacktraces/git info armbian-next: introduce `POOR_MAN_PROFILER=yes` (under `ANSI_COLOR=none`) armbian-next: `docker`: fix Dockerfile output indentation & warning msgs about generation armbian-next: make sure temporary `kernel` and `u-boot` CLI commands always build a kernel/u-boot even if cached `risc-v`: correctly include `debian-ports-archive-keyring` in Risc-V CLI packages `risc-v`/`starfive`: use mainline kernel 6.1.y, with StarFive's rebased patches against `v6.1.5` `risc-v`/`starfive`: update kernel config, sans changes `risc-v`/`starfive`: `CONFIG_MOTORCOMM_PHY=m` for the onboard Ethernet `risc-v`/`starfive`: switch kernel config to `savedefconfig`'s output, cos that's what it should be `odroidm1`: enable working userland `fw_setenv`, so users can go back to Petitboot & otherwise tweak u-boot env from Armbian `odroidm1`: bump to 6.2-rc3; look ma, no patches. `odroidhc4`: pre-configure `fancontrol` so fans work right out of the box armbian-next: debug note about fstab with tmpfs in chroot armbian-next: mark Vagrant CLI as unimplemented armbian-next: `docker`: use dash-lines before/after launching docker; set `ARMBIAN_INSIDE_DOCKERFILE_BUILD=yes` when it is so armbian-next: `traps`: allow for cleanup handlers with arguments; unify around `run_one_cleanup_handler()` armbian-next: `ccache`: show more in ccache debugging armbian-next: make superglobals readonly (`DEST/WORKDIR/LOGDIR/SDCARD/MOUNT` and others) armbian-next: introduce `tmpfs-utils.sh`; put `LOGDIR` and `WORKDIR` under tmpfs; set `CCACHE_TEMPDIR` under `WORKDIR` armbian-next: split `prepare_host()`, fix `.tmp` reset trap armbian-next: do basic host checks before config if interactive, or during prepare-host if not armbian-next: tmpfs-utils: last-resort debugging, to stderr, of failed tmpfs unmount armbian-next: logging: squash nasty leaking file descriptor bug due to "tee" armbian-next: bring back `swig` hostdep -- needed for some u-boot's armbian-next: severe bug, countdown counted up armbian-next: kernel: cleanup bundle after patching succeeded, not after build success armbian-next: firmware: don't build `-full` firmware if not on CI/noninteractive and board's not going to use it armbian-next: `chroot_sdcard_apt_get_update()` to replace and better log `chroot_sdcard_apt_get update` armbian-next: extensions: rename `kernel-localmodconfig` to just `lsmod` armbian-next: docker: remove old code armbian-next: try harder to get `en_US.UTF-8` logging armbian-next: drop dead/duplicated code for bootsplash and mark kernel-drivers.sh (which includes bootsplash) as dead code armbian-next: extract `move_images_to_final_destination()` and optimize for fast-move when src/dst on same filesystem armbian-next: tune logging: unmounting, u-boot prep/source/install; traps; extensions armbian-next: don't leak `.raw` image if failure before move; show rootfs-tmpfs info armbian-next: `mktemp` and `tmpfs` related utility functions with automatic cleanup handlers armbian-next: small cleanups: squash typos / add function keyword / add types / mark dead code armbian-next: small cleanups: squash typos / add function keyword / add types / mark dead code armbian-next: add logging with size of actual built rootfs in MiB; debug tmpfs in trap & after pkgs done armbian-next: still fighting `tee` leaking under duress, and I think I won armbian-next: `compile_firmware()`: use new temp dir helpers (saves 2Gb+ in WORKDIR), fix typos armbian-next: severe bug, there was no prefix-emoji for Windows armbian-next: allow shortcut `EXT=` to the overly-long and plural `ENABLE_EXTENSIONS=` armbian-next: use retries for downloading ORAS tooling WiP: aggregation.py vs create-cache.sh: better hashing, much desktop WiP: patching.py vs decent logging: better, but not there yet ------------------------ armbian-next END marker ----------------------------- Richard Neese <r.neese@gmail.com> (1): `risc-v`: switch from `grub` to `extlinux`; add Debian Ports keyring hostdep to strap Debian riscv64 from ports Tony <tonymckahan@gmail.com> (1): armbian-next: officially support WSL2; pester user for UTF-8 terminal
-
18 hours ago, schwar3kat said:
I'm guessing that ">" on the end of the line in the logs means that the patch had no issues and applied correctly?
Err... no, current patching log is a terrible mess and very hard to tell wtf is going on.
I am making a complete review of the patching logging. Objective is to produce a simple "table-like" output like we had before, on screen; then have details/rejects/etc on the logfile when investigating.
Sorry for the bad experience, it's gonna be much better "real soon now".
PS: For now, the least-horrible way to see the Patching info is looking at the produced .md file in output/logs. Render it to UTF-8 HTML for full glory.
-
3 hours ago, rpardini said:
So it can't get worse
A lie. I just found a bug where I mangled perfectly valid UTF-8, during mbox parsing, for no good reason. Fixing...
-
2 hours ago, schwar3kat said:
The problem here might be that the Chinese in the table like structures might have a purpose (print or error strings come to mind).
Yeah, parsing the patches includes being able to decode them.
Patches should match the encoding of the target patched source; In this case, it's UTF-8.
In my view, those patches are mangled (not just "in big5"): the original encoding was lost due to copy/pasting, and they're already (in master) putting frakked up characters in the source code.
So it can't get worse, but now I warn you about the error.
-
3 hours ago, balbes150 said:
It suits me (joke).
It suits a lot of people -- not only you.
I am (seriously) taking in the sensible parts of your feedback and implementing. (it would be easier if you were not also insulting, I get distracted by the insults when parsing your text).
I already got most of the "junk" out ("junkless", for the "Oleg" scenario: fully native arm64 assembly, no Docker) while still keeping the capacity to have a "junkfull" Docker image with everything (without having to sync 2 pieces). It's _a lot_ of if's.
Next, (pun!), I will handle the logging of the patch apply to output what we had before (just patch name, OK or not, colors), and hide the details in the logfile. (eg; rejects, etc).
Then reduce the logging to file of things we don't need & fix the formatting of the single .txt log as has already been discussed a few pages back.
Also https://armbian.atlassian.net/browse/AR-1451 so logs actually show up on screen.
So it's just more code / more if's -- and more scenarios to test.
The whole reason I wanted to avoid _adding more code_ is that the more code I add, the more code to review, and the longer it will take to merge.
But I'd rather have everyone onboard, or, if that's not possible, have most people onboard, even if takes a bit longer.
-
1 hour ago, going said:
1) I don't need the source packages.
Definitely not for Kernel.
Consider a new user, that wants to patch something really simple, for example, add a missing LED to a board via DT. (that's how I got started.)
What is easier / more productive: download kernel-source-xxx package, make it work somehow, see a complete source without any history/commits, not understand why things are done that way... etc.
Or: go to armbian/build repo, see history & documentation & happy friendly people collaborating & a tool that guides them to produce a new build & a good place to PR his small patch so that it lives later....
-
2 hours ago, schwar3kat said:
The build process should strip out the diff's and process them.
There should be no checking of mbox formats at this point because the builder is not responsible for all other developers patches.Take a look at `patching.py` if you can muster, for context.
I am trying to strike a difficult balance: if the file passes a few tests (matching magic markers, unidiff well-formed, others) then I handle it as an mbox, and split them.
In certain conditions, as you mentioned, this fails, as is the case with heavily-mangled files, and the build refuses to continue, as an indication of "the bar has been raised, and xxxx patch doesn't pass".
In any case, the actual patches are being applied using the `patch` utility like always (so allow for fuzzy/offset, etc), and if it turns out half-decent patches is "impossible" (it isn't...) the code can be made to not split, and will then behave just like before.
We could do that per-family, especially for legacy stuff no-one wants to or will fix.
So let's strive to clean -- it's really not that hard (or what am I missing?).
Later, we can add GitHub PR-level automatic checks for well-formed patches, thus "keeping" it clean for the future.
-
2 minutes ago, Igor said:
Someone has a better idea?
Yes, I have already started work on `armbian-oleg` -- it only builds media, only on arm64, and does not log anything.
-
1 minute ago, Igor said:
Oleg builds his things on arm64, but we have to cover more, perhaps some stupid things for legacy reasons. We can't cut off blindly. I am totally aware that we are dragging 3/4 of those without any point, but someone has to remove them and be sure nothing is broken ... but since this falls off with the NEXT, why wasting time dealing with that?
Lets focus into seeking for solutions. And compromises. I am sure we will need to make some - our common goal is have the tool maintainable and modernised. And this is not a fixed thing - when better ways are found and if most of us agree that something can be removed, we remove. If not, its better to stay. There are people out there, that are using this build framework. We don't want to cause stress to them.
I see. Let's then focus 100% energy of the whole team to cater to Oleg's specific wishes and needs until he is satisfied.
Also, let's give him his own family, let him arbitrary fuck up and duplicate code for no reason, enable Tegra on Rockchip family, etc etc, and then in the end complain. When does it stop?
-
55 minutes ago, going said:
ib/functions/host/prepare-host.sh
Yes, only if amd64. Oleg is about "only arm64".
I will thus stop all development and start endlessly bitching about "junk" and being useless. -
3 hours ago, balbes150 said:
Now this is implemented in MASTER, do not load cross compilers if the native assembly is running
You are delusional. `master` downloads & extracts 9gb of multiple different versions of Linaro etc toolchains before it eats any breakfast.
-
14 hours ago, schwar3kat said:
Summary of actions that were required to prevent errors from aborting the build:
Awesome. I just pushed `8a2f30f4a1d648c4e15df5944241e80847bc8f78` with:
14 hours ago, schwar3kat said:This behaves like current armbian-build and ignores errors caused by ntpd using the port.
Fixed; I check if the pidfile for ntpd is there, and if so, skip it completely, saving us 2 seconds. Even if not skipped, I added the "|| true" so it won't fail for any reason. https://github.com/armbian/build/commit/d42d340f48848126abb571b389b65d68a1732da4
14 hours ago, schwar3kat said:freespace
Fixed (hopefully), I just added "|| true" -- there's work going on in master to avoid the double-mount and other bugs that might appear here, I'll wait for them. https://github.com/armbian/build/commit/e84d32785f8de4eab4763c120bd4c367d4ee6741
14 hours ago, schwar3kat said:commit.gpgsign false
Added as a command-line option to the "git ... commit" invocation at that stage, so shouldn't fail anymore even if you have it enabled globally. https://github.com/armbian/build/commit/96926fbf4f8a1cb96dc18e4a4411f9fc3c594bec
14 hours ago, schwar3kat said:a problem with something cached causing makefile corruption.
This was due to a borked merge coming from master, was up for a few hours, sorry about this. (ref extrawifi)
8 hours ago, schwar3kat said:I tried building with BUILD_DESKTOP=yes.
I expected a dialog to select which desktop and base config, but it aborted with an error.Fixed. Another case of "let's ask the user" in the middle of configuration -- I had forgotten this. https://github.com/armbian/build/commit/1e53ac23a84c9135394dead7a0bf5ca8f8dfbf58
Please notice there's a known issue with desktop rootfs caching I am investigating.
Super thanks @schwar3kat for all the feedback and patience.
-
2 hours ago, going said:
The target for all these features should be the source package
Hmm. Yes. This is how buildd / debuild / Launchpad works.
You create a source package (a .dsc, .changes file) including the pristine source in a .tar, then the "debian" packaging source.
Then you build it with "debuild" or "dpkg-buildpackage".
This would be great, but is very different from what we do now, won't support cross-compilation, and thus requiring native iron to build any package.
It also requires release-specific packages, even for thing that aren't really release-specific (say, firmware).
So yes, this is maybe for 2024.
-
2 hours ago, balbes150 said:
I haven't seen such shit yet,
Great, very mature. Congratulations. Keep adding value by doing this, instead of writing 2-3 lines of text contributing something.
-
On 1/8/2023 at 8:34 AM, going said:
I do not approve of the presence of any script automating the patch conversion process in the build system itself. This can only be as an additional utility designed exclusively for manual use.
Yes. It is a process that requires human. What I automated is the boring part, so the human only has to use the eye at the end of it. At least for trivial rebases.
It is part of the Python patching system, but only used if invoked manually by a developer. Don't worry.
-
13 hours ago, schwar3kat said:
The patch parser doesn't appear to be able to parse a patch with a mbox conversation history.
The attached original file is not one of mine. It is bad and has format issues resolved as follows:I moved the first three lines comment into the email body.
The patch is still messy and created warnings on the checkpatch.pl test but it should have worked.
I had to disguise the mbox history except for the last email before it would parse the patch.
I disguised the history email headers by adding a "-" to the beginning of each line.Hmm. This (damn) patch is everywhere. Case in point: https://github.com/armbian/build/commit/3790d46c6dbb3b7414d68109ebaaa93dd69bc207 and https://github.com/armbian/build/commit/d89b9cb3641fc119a72a11ea32fd5327578d5e9c
That is simply a complete mess of a single mbox, it's not "conversation history", it's just a complete mangle.
Worse, it was copied around everywhere in many families.
We've also a lot of patches with invalid utf-8 chars due to mangled copy/paste of chinese code, also copied around, then modified differently in each family.
Cleanup is necessary for sanity, most of those things should just be dropped.
11 hours ago, going said:Then add the modified files to track the git git add -A. And make a commit git gui. In the commit message, add all the text that you consider important.
Then extract the commit using the git format-patch -1 commandI have this process fully automated in the new patcher, including keeping certain matches in the same file, and rewriting both in-place (rebase against kernel built) and via git (for doing rebase across versions).
There will be a CLI for this very soon. "kernel-patches-to-git FAMILY=xxx BRANCH=xxx" and "kernel-rewrite-patches FAMILY=xx BRANCH=xx" or something like it.
-
3 hours ago, schwar3kat said:
I ran the build again, and it apparently didn't pick up the change. It skipped the kernel build and used the cached kernel.
I had to delete the deb file to get it to build the kernel.Hmm, I don't think I changed anything regarding this -- yet.
Currently the patches-hash and the real (6.1.3 etc) version of the kernel is not included in the deb filename/version. (it's static, "trunk123" or "23.02" etc).
And if the deb already exists, the build is skipped.
Hopefully soon we will include the patches-hash and the kernel version in the deb/filename, so the build detects the .deb is out of date and builds again.
-
2 hours ago, schwar3kat said:
I guess some new parsing ideas may have to evolve.
The gold standard (conveniently written in Python) previously known as "get-lore-mbox", now b4: https://b4.docs.kernel.org/
Its output, taken from a lore link, is what we should aspire to have.
b4 am -o- "https://lore.kernel.org/all/20220701185959.GC1700@begut/"
For Armbian specific case, though, I'd like to increase unidiff size from 3 to 10 by default -- at least on .dts / .dts files to address at least most of those pesky "} } }" dts blocks endings mispatchings.
-
1 hour ago, going said:
What will show if replaced with this code:
Those freespace checks can fail in subtle ways, also in master, it's hell, looky this https://github.com/armbian/build/pull/4656
Armbian developers meeting 2/1/2023
in Armbian Project Administration
Posted
This week's shortlog. Really short, this time, since most of the artifacts stuff isn't included yet.