Jump to content

rpardini

Members
  • Posts

    132
  • Joined

  • Last visited

Posts posted by rpardini

  1. @c0rnelius graciously submitted GitHub PR to upgrade u-boot that effectively removed the patch in question from the equation for the Radxa Zero.

    While it was nice to "boot USB first", since it enabled a few interesting scenarios (boot non-Armbian EFI OSes in USB disk), it's not worth having it if normal Armbian operation is impacted.

    I've advocated for this usb-first style to be default around u-boot 2022, but we're now moving to board-specific support (and removing completely in the case of the Zero) for 2023+.

     

  2. 3 hours ago, schwar3kat said:

    this change tagged as [HIGHLY CONTENTIOUS CHANGE].
    @rpardini  do you have any insight into this change or advice?

     

    Yeah that. Well, cpufrequtils and big.little clusters don't mix well as you probably noticed.

    cpufrequtils itself has been deprecated for well over a decade.

    Since cpufrequtils was being inflicted on otherwise perfectly working, DT-controlled boards, we've disabled it by default.

    Boards that are proven to require it can re-enable it directly on the board as you've already found out or in the Armbian board file via 'CPUFREQUTILS_ENABLE=true'

    All that said, the real, kernel-supported alternative, cpupower, requires kernel-tools build which is currently not provided by Armbian (nor is usbipd, perf, nor any other kernel userspace tool, due to the complexities of release-dependent building of parts of kernel which is normally userspace-agnostic).

     

  3. On 9/28/2023 at 9:01 AM, jglathe said:

    The armbian build framework does create an image with lunar, the final stages of setup balk on wrong rights and essentially lead to an incomplete install

     

    Please build with `SHARE_LOG=yes` and send us the URL so we can check.

     

    On 9/28/2023 at 9:01 AM, jglathe said:

    it is coming up with battmgr working

     

    That's due to the pd-mapper and qrtr in userspace working. (Only for lunar, coming from ubuntu x13s ppa)

     

    On 9/28/2023 at 9:01 AM, jglathe said:

    Touchscreen working (yay again!)

     

    Yep it has a hid bind loop that is work-arounded via some udev rules... it's strange.

     

    On 9/28/2023 at 9:01 AM, jglathe said:

    Would the wdk2023 be a candidate target for armbian, too? Do I just create a microsoft-dev-kit-2023.wip and on it goes?

     

    Mostly, yes. I'd say send your patches to steev, or add them to Armbian. Copy the x13s board file and change the DTB.

    If this works we can refactor/unify more later...

     

    On 9/27/2023 at 9:34 PM, jglathe said:

    This is the version I hacked up for the x13s, which appears to load the desired firmwares as intended:

     

     

    Yeah take a look at the x13s board file, you'll find something really similar. I have not kept up to the rapid amount of changes after 6.3.y. I might try to update to 6.6, since that's gonna be LTS.

     

     

  4. 1 minute ago, emk2203 said:

    Only missing thing for me would be this image based on an Ubuntu userland, but it's just habit. Bookworm is not lacking in comparison.

     

    Definitely pick the PR and build whatever userspace you need, there's not much that is userspace dependent (maybe mesa, but I've not tested desktop usage). I've tested on Bookworm, Jammy and Lunar, all work great.

     

    5 minutes ago, emk2203 said:

    Basically everything from my wish list satisfied

     

    Also: UMS mode works absolutely great. Use an SD + recovery button to deinfest, after that you can stop u-boot with the serial console and for example run "pci enum; nvme scan; ums 0 nvme 0" and be able to flash to NVMe via UMS (usb otg) without any SD or eMMC.

    Also2: you can fix the MAC address stored in the u-boot env in SPI to match the board sticker if you want. Otherwise, MAC address is generated based on CPU serial number, which is stable, but does not match what's on the sticker. Dunno where HK stores the sticker value (maybe some fuse/OTP?)

     

  5. https://github.com/armbian/build/pull/5606

     

    Testing image:

    https://github.com/rpardini/armbian-release/releases/download/23.08.18-rpardini-434/Armbian_23.08.18-rpardini-434_Odroidm1_bookworm_edge_6.5.0-rc6.img.xz

     

    #### `odroidm1`: de-infest Petitboot 🔥 use Kwiboo's 2023.10 u-boot; UMS works; bump kernel to 6.5-rcX

    - `odroidm1`/`edge` (`rk3568-odroid`): bump to 6.4.y, update config, rebase patches to 6.4.10
    - `odroidm1`/`edge` (`rk3568-odroid`): bump to 6.5-rc6; manually fix RK808 breakage in .config; externalize overlays; use Makefile autopatcher; rebase patches
    - `odroidm1`/`edge` (`rk3568-odroid`): drop 6.3 and 6.4 patches
    - `odroidm1`: de-infest Petitboot 🔥 use Kwiboo's 2023.10 u-boot; UMS works
      - using Kwiboo's `rk3568-2023.10` branch  with BINMAN-handled blobs
      - patches (defconfig unless indicated):
        - boot usb first (rockchip-common)
        - blink leds & keep red one one on preboot; reset SPI env once after deinfesting from Petitboot
        - change usb_host0_xhci to otg (u-boot dtsi)
        - enable DM_GADGET, UMS 🔥 and RockUSB
      - **usage instructions**:
        - build & burn image to SD card
        - insert SD card into board
        - **hold the recovery (RCY) button** and power on the board
        - watch board boot
        - **de-infest Petitboot**: use `armbian-install` to install bootloader to MTD
          - if you don't, you'll need to hold the recovery button every boot
          - optionally: use `armbian-install` to install OS to eMMC/NVMe/USB
        - power-off board
        - remove SD card (new u-boot always boots SD first!)
        - boot into your newly de-infested machine
          - boot order: USB, SD, MMC, NVME, SCSI
      - de-infested machine can now boot (directly) from USB/SATA/NVMe, possibly via EFI:
        - Armbian UEFI-arm64
        - Fedora 38 aarch64
        - HASSOS for ODROID-M1
        - Talos arm64
        - others...
      - extra: new u-boot by Kwiboo (with GMAC patches) gives us stable MAC address
        - although it is based on cpuid#, doesn't match the HK sticker on the board
          - run `fw_setenv ethaddr XX:XX:XX:XX:XX:XX` to change eth addr in SPI flash environment if needed
      - `odroidm1`: update DDR/BL31 blobs (this depends on https://github.com/armbian/rkbin/pull/20)

  6. On 6/20/2023 at 7:13 PM, Erica said:

    What else can I try? 

     

    Hey @Erica please try this experimental image: https://github.com/rpardini/armbian-release/releases/download/latest-images/Armbian_20230600_Khadas-vim3_bookworm_edge_6.4.0-rc5.img.xz

     

    Instructions, for any VIM3, in any state (middle button 3 times in 1 second tells the MCU to force the SoC to boot from SD/eMMC).


    -> Write oowow (https://dl.khadas.com/products/vim3/firmware/oowow/vim3-oowow-latest-sd.img.gz) to SD card. Insert SD card with oowow.
    -> Power on board, press the middle button 3 times + reset until it reboots into oowow.
    -> Use oowow to clear the SPI ("fast" method is enough) *and* the eMMC ("fast" method is enough).
    -> Use oowow to set "bootmode" to SPI. (This way it will try SPI, which is empty, then eMMC, which is also empty, then SD).
    -> Power off the board (important. MCU changes don't take effect unless you power it off completely).
    -> Write Armbian to SD card, insert the SD card with Armbian.
    -> Power on the board. Boot Armbian from SD. Use `armbian-install` after preparing the partitions on the target device (eMMC/NVMe/USB).

     

    Good luck.

  7. User on Discord reported problems using the -rt patches with the main branch on Amlogic edge.

    1) RT patches are specific to certain linux versions, and the fact most of our families use say 6.2.y branch (and we can't easily override KERNELBRANCH= due to the tangle between board/boardfamily/family_common source order conundra). Would need to either hack at the meson64_common file to change it to to "tag:6.2" to use the 6.2.0 release, or use a hook in a config file to to the sam. Suggestion: add a FORCE_KERNELBRANCH param to work around this more easily for users without requiring a config file change.

    2) RT patches add in a ./localversion-rt file, which is picked up by Kconfig and included in CONFIG_LOCALVERSION, which changes the paths of _everything_ from what Armbian expects. Say from "6.2.14-meson64" to "6.2.14-rt3-meson64". The kernel builds, but packaging gags. It's possible to remove the patch that adds that file so the version matches, but Armbian definitely should be able to handle this more gracefully, either by automatically removing "localversion-*" files, or by actually honoring them.

     

     

  8. On 4/13/2023 at 10:38 AM, Igor said:

    release numbering changed from 23.05 -> 23.5

     

    Sorry Igor I missed this during the call.

    This might... lead to problems, since we lose numerical sort comparison if we do this.

    Say "230501" and "230602" and "231000" and "240200" all sort correctly.

    But "23501" and "23602" and "231000"  and "24200" are... broken.

     

  9. On 4/1/2023 at 3:38 PM, usual user said:

    They are floating around

     

    Thanks for the report. For a minute I thought something more radical had happened, but it seems situation is mostly the same. In the end, it's about SPI -> NVMe boot, possibly with basic EFI support, that would shift the status quo.

    For now it seems "interested-enough" users can manage to make pboot boot the Armbian image and make it work decently well for most server/NAS/etc needs.

    We do have a few outstanding issues, like Eth MAC being random, any idea about that? (usually also at u-boot level, but I haven't looked into how HK does things)

     

  10. On 3/24/2023 at 6:19 PM, Werner said:

    Maybe @rpardini can shed some light here?

     

    I'm happy with the current status of it, using HK's blobs / SPL / u-boot from factory SPI. 

    I've added userspace tools to fiddle with u-boot env in SPI which makes it very usable.

    Kernel is pure mainline, someone contributed overlays (which work).

    @usual user has posted new stuff with mainline u-boot 23.0x -- if that ever gets nvme boot support I'll be investigating so we can make a full "de-petitboot-ified" build, including writing mainline u-boot to SPI. that's for later.

    @usual user will probably also know about RGA and other stuff....

  11. On 3/11/2023 at 9:10 PM, SteeMan said:

    For example on the local cache the hash could be the parent directory name.  So instead of:

    ../debs/linux-image-current-meson64_6.1.13-S1ac8-De511-Pb8ce-C072bHfe66-B60c8_arm64.deb

    it could be:

    ../debs/S1ac8-De511-Pb8ce-C072bHfe66-B60c8/linux-image-current-meson64_6.1.13_arm64.deb

    This leaves the .deb package format/name independent from the caching mechanism.  Thus separating issues 2 and 3 above.

     

    Hmm, not really. Separating the local cache only solves the problems involved in local caching, but the real hard part is solving for repo (apt repo, apt.armbian.com and beta variant).

    That simply can't be separated by folders, and usually only a single version of any given package is allowed in any single repo.

    Thus the `Version: ` field (which determines the .deb filename too) is the bottleneck here, and directory tricks won't work.

    But yeah, I very highly appreciate your try to detangle this stuff -- it is very, very hard.

     

    I am moving on with the prefixing implementation, and to hell with cache hit ratio, at least for now -- we've many things breaking cos an old version from repo is being considered "newer" than a just-built local .deb. (eg: installing armbian-config brings in sunxi-tools which causes an "upgrade" to linux-image-sunxi-edge which is actually older than just-built. it's insane.)

  12. 2 hours ago, Igor said:

    should we get rid of the DT package and place DT blobs to the same /boot/ location as in DT package (as done in media kernel)

     

    Well this is a completely different subject, but yeah, valid. That probably/finally explains why media had double-patches to mkdebian/builddeb (which was already patched/replaced by Armbian before patches applied). I was trying to understand that, for months, and failed. Not a single comment explained it, as seems to be Oleg's policy. I will revisit it now that I know wth is going on and maybe it will make sense, but sincerely...

     

    Currently and before:

    - linux-image: has kernel, modules, and DTs (!), (in /usr/lib/<kernel_id>) -- as expected by standard Debian tooling (u-boot-menu, flash-kernel, etc). We had a long discussion about this, 2 years ago. Did we forget? (No, it was just Oleg said he agreed, but ignored it and patched his own stuff "secretly" sans any explanation or discussion).

    - linux-dtb: has DTs in /boot (as expected by "most" Armbian families, using u-boot and bootscripts, also the armbian-install, config, overlays, etc).

     

    Either way, we can definitely accommodate this "single pkg" need, but I'd like to do it in a way that is 1) comprehensible 2) usable by more families than just one 3) not a maintenance nightmare, eg, NOT double-patching. I don't think that is unreasonable.

     

    So if we could spell out what is the need, it would be easier... again: let's work together. we've 2 months to figure out stuff before release. 


    I understand now that it seems that I just blindly changed the way `media` works and am crazy... might even be true, but, we can fix, and would already have fixed if more engineer talk and less drama. IMHO.

  13. 41 minutes ago, belegdol said:

    I think we might need a new plan.

     

    Yes. I've remembered that adeepv/adeepn (Viacheslav) had something related: a privately hosted, pure-git kernel, to which we couldn't get HTTP access at all. In that case, including the base_version (eg v6.2.1) is not possible. Both for that case, and for this "HK's 5.4.y xu4 + incremental patches", we might introduce something like "KERNEL_SKIP_MAKEFILE_VERSION=yes" to be configured in family file (and _not_ FORCE_KERNEL_EXACT_VERSION_VIA_PATCHES like I mentioned before). Together with the "add $REVISION prefix to all artifact versions", we'd just not have the 6.2.1 version component, and instead just use the "S"HA1 of the commit (which is always available). This should also avoid the "forgot to update" misleading problem we've identified above. And it solves 2 problems...

  14. Hello folks, sorry for the absence, I've been facing connection issues.

    Let me try and explain a bit, from the beginning. I will focus on kernel, but most applies also to u-boot, firmware, etc; also keep in mind that not all packages have been migrated to the new scheme: BSPs, Desktop BSPs, etc, are not yet changed (too many things to hash / little benefit / too many things to decide before that).

     

    The reason we've a new version scheme is for consistent caching and fast CI runs.

    The important part about the caching is that we need to obtain the "desired" version of the kernel, **before** actually git clone/fetching the sources, before patching, and before compiling; otherwise it's useless: if the user/developer has to do any/all of this to get to the version, then it's "too late" for caching. This needs to be crystal clear, otherwise everything just looks like an insane person did random stuff and broke Versioning for no reason.

     

    Important: the caching part benefits some users/builders, in the sense that someone casually building an image at home might hit the cache, but the main target/reason it exists is for our (Armbian) own Continuous Integration (CI): every PR, every nightly, every release, etc, should benefit from the caching, but it has to be consistent (not use wrong version), and hit ratios should be high. Before this scheme, we had only REPOSITORY_INSTALL mechanism, which frequently installed wrong/mismatched versions, we just said "use whatever is in the apt repo", which could be literally anything, and frequently led to half-working images.

     

    Also, "apt repo" is a very inflexible/demanding solution in terms of infrastructure, which requires HTTPS servers/mirrors, Release lists, etc, and is not available for free for every developer/partner. New caching uses standard OCI registries, which are available for anyone to host, and even hosted for free on GitHub (ghcr.io) and DockerHub among others.

     

    That said, let's get to how we calculate the version for the kernel package. The code, containing a long explanation, is at https://github.com/armbian/build/blob/main/lib/functions/artifacts/artifact-kernel.sh - but in essence goes like this:

     

    1) "base_version" (bv): Get the KERNELSOURCE and KERNELBRANCH variables (usually set in family config file for the board being built). Use `git ls-remote` to obtain the SHA1 of the commit of that. Obtain the HTTP URL for the Makefile at that commit. ("View raw" in GitHub web interface, but also for GitLab, and others). Get the Makefile contents (using HTTP, not Git, since there is not "partial git fetch"). Parse the Makefile for the version information inside, including the "codename". (Codename here is just to "prove" that data comes from Makefile, not "git tag"). Cache all this on disk for a few minutes.

    2) "S"ha1: just the SHA1 of the commit mentioned above. 

    3) "D"rivers: Obtain a SHA256 hash of the "drivers". Kernel drivers (EXTRAWIFI etc) are just a complex variation of patches; they are shared across families, and are not stored as patches, but in the end are transformed into one huge patch, so behave like one.

    4) "P"atches: Obtain a SHA256 hash of the "patches": simple hashing of the patches/series/etc under KERNELPATCHDIR (and userpatches variations). If no patches, "0000" is used.

    5a) "C"onfig: Obtain a SHA256 hash of the ".config" file that is gonna be used for the build. LINUXCONFIG (and userpatches variations).

    5b) "H"ooks: Obtain a SHA256 hash of the hooks that might modify the .config. (This is a bit more complex to explain, since it involves extension mechanism, which is out of scope here). Suffice to say that if you change a hook that changes kernel configuration, the version should change.

    6) "B"ash: Some lib/framework code, if changed, should cause the version to change as well. Not all bash code, but bash that calls the "make", or does the packaging, if changed, need to change the version. Currently "lib/compilation/kernel*.sh" is hashed.

     

    Hopefully, you can now understand that `Version: ` below. Each SHA256 hash is shortened to a mere 4 characters:

     

    On 3/5/2023 at 3:58 PM, belegdol said:
    Version: 5.4.228-S3043-De511-P2c81-C951fH6842-Bb436

     

     

    So now we've to address two things: a) the original question by @belegdol; b) the general question of "how to control upgrades via apt".

     

    ------------------------------------------------------------------------

     

    Ref the "odroidxu4" situation: @belegdol maintains that kernel family, in the following way (please correct me if I'm wrong):

    a) Vendor, HK, has an  Git repo, which has a 5.4.y version, plus HK/Exynos patches. That Makefile currently says "5.4.228". It has heavily lagged behind official releases in the past AFAIK but seems pretty recent recently. HK "merges" updates from mainline, instead of cleanly rebasing, though, since they've lost all hope of mainlining their stuff and merging is easier.

    b) Belegdol extracts new/recent 5.4.y patches from upstream/kernel.org, (possibly?) manually fixes them so they apply cleanly on top of HK kernel, adds them to Armbian via PRs. Those patches modify the Makefile so the version is incremented.

     

    So AFAIK this is the only kernel family that is done this way (we did some similar stuff for rk3588 legacy, but thankfully no more). Every other family works in reverse: we start with a pristine/clean/updated kernel.org branch (whose Makefile is updated), and then patch what is needed on top of that. That way the kernels "auto update", unless some patch breaks.

     

    Hopefully it is clear, together with the above explanation, why the  "wrong" version is used: only the original Git's Makefile version (not tag... but tag's Makefile's contents) is considered, since patches are only ever applied after Version is calculated, and cache possibly hit or missed.

     

    So while I confess I had not xu4 as the primary target of this whole thing, it representing less than 1% of all boards. Still, there needs to be a way to make it work, we just need to decide how.

     

    One possibility: invert the XU4 patching scheme, by rebasing HK's branch onto an upstream tag, and extracting the patches (git format-patch); then change our KERNELSOURCE/KERNELBRANCH to mainline, and let it auto update. This might, or not, be a lot of work initially, and might, or not, impose a lot less work over time. It depends on how many changes there are in HK Git, if the relevant patches are constantly updated, etc. From what I hear HK has little work done on the Exynos, but I am not an expert. I would like this scheme since it would align XU4 with the rest.

     

    Another possibility: introduce a way to (optionally) override the "base_version" (example, variable FORCE_KERNEL_EXACT_VERSION_VIA_PATCHES="5.4.232") directly in the family configuration, use that for the artifact version if present, and change nothing else. This is _more_ work for xu4 maintainer, and might be get out-of-sync with reality (change patches, but not change variable, and vice versa). This might be useful also for other situations, although none come to mind right now, it might be useful for Igor's "emergencies" 😉

     

    ------------------------------------------------------------------------

     

    Now about "how to control upgrades via apt". 

     

    To understand this, we need to understand the formally defined Debian Policy about the "Version: " field in the DEBIAN/control file.

    I recommend we all read https://www.debian.org/doc/debian-policy/ch-controlfields.html#s-f-version -- it is one of the more dense, complex, and important paragraphs of text ever written by Debian, and defines how apt decides what version is "higher" than another.

    There, we can see there are 3 components, each with its own rules, and the special rule about the "~" (tilde) character.

     

    The important part: I did not want to "impose" the final apt versioning scheme before we talked about it.

    Also: there's pending work in the general area of "repository management/publishing" that needs to be done before we can publish _anything_, consistent or not, to any apt repository

    I was told we had until May/2023 (so circa 2 months) to reach consensus and implement those -- but now it seems the sky is falling down much earlier?

     

    So current "Version: " is a non-working, non-deterministic, inconsistent situation in regards to apt upgrades: if you try to predict how our "bv+S+D+P+CH+B" will be handled by apt, you will soon realize it's inconsistent: "fixed"/later patches on same base version might get downgraded to broken ones, and vice versa, almost randomly (as is the property of SHA256 hashes).

     

    One extremely easy way to "fix it" is to prefix our current "bv+S+D+P+CH+B" with the $REVISION (which comes from VERSION file or equivalent userpatches) and a tilde.

    That would force apt to respect the "old" versioning scheme, while still keeping caching consistent.

    Unfortunately, doing that would drastically decrease cache hit ratio, since changing VERSION file would cause an otherwise identical kernel to be ignored/rebuilt. That might happen because user/dev changed userpatches/VERSION, but is otherwise identical to main, or because we branched off a release/tag (otherwise identical to main at that instant).

     

    But it might be the best option right now, especially if no-one else is willing to invest brain time on this. I could then twist my own brain on how to recover the cache hit ratio somehow else (eg, search cache for both with-$REVISION and without-$REVISION variants, invent some "IS_RELEASE=1" variable, doing GitHub Actions acrobatics, etc; is a lonely world where only Igor and Ricardo live).

     

    Similar option: transforming $REVISION ("23.05.01") into an integer (230501) and using that as the "epoch" part of Version field, would also work, but same cache-non-hitness applies.

     

    -----

     

    Phew, that was long. If you read all this, give yourself a pat in the back. Let me know what you think on all the different points... @going @Werner @belegdol @Igor @SteeManand everyone else interested.

     

  15. 6 hours ago, schwar3kat said:

    I usually set global git username and user email. 

     

    This has been handled for other cases where `git format-patch` was used (kernel drivers cache, I think) so should also be fixed: it should use MAINTAINER and MAINTAINEREMAIL (default to Igor) if not set on git config. git config might not exist at all for example if using Docker -- so this is smth that should be handled anyway. Can you open a GitHub issue? I am losing track of stuff by splitting between forum and GH....

  16. 34 minutes ago, schwar3kat said:

    New main branch.   Error on first run.  The build works on second run.

     

    34 minutes ago, schwar3kat said:
    Downloading required [ ORAS tooling ]

     

    Thanks @schwar3kat ! This has been elusive, but its clear that when the tooling is 1st downloaded, the parsing fails.

    2nd run the tooling is already there so doesnt fail. 

    Will try to address!

×
×
  • Create New...

Important Information

Terms of Use - Privacy Policy - Guidelines