rpardini

Ok, a weekend later, here's armbian-next v64. https://github.com/armbian/build/commits/armbian-next - do `git pull --rebase` and build away and report problems here 😉 Builds are green. https://github.com/rpardini/armbian-release/actions/runs/2289642908 Images are up for download: https://github.com/rpardini/armbian-release/releases/tag/20220508d (a few novelties there. eg, ODROID-M1) ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- No fun in that @Igor😉 -- I've fixed the Makefiles for cubie-i and clearfog u-boot so they build. Those were exactly the same problem as from xu4: 2017/2018 u-boot, that original Makefile wanted "armv5" target which does not exist anymore in newer gccs. https://github.com/armbian/build/commit/0a3fc81b5e519a11ac41c1d196b076a91ce38a1b Most of those boards are actually armv7a, so I patched and they built. If they _work_, we'll only know if some of you download the above images and try it out (cubie, clearfog, odroidxu4 at least). Thanks! Many other changes/fixes and a few breaking ones, like removal of LIB_TAG/.ignore_changes for @lanefu I've not tested desktop building so here's when @Rich Neese comes in, let me know. Thanks for all the tests!

I've some images, but they're really quite far from usable right now. Vendor kernel 4.19 I can't get HDMI to work. Everything else works. I'm using it to build Armbian itself and works great with NVMe and the 8gb RAM. Mainline 5.18-rcX has working HDMI, but no NaNeng PHY, so no PCIe/NVMe, no USB3, and no SATA. tobetter is working, also work in quartz64a and other rk3586/8 will benefit this. nand-sata-install will not work with this unless I find a way to flash uboot to SPI (and this remove petitboot, but there is no official petitboot recovery from HK yet) and mainline u-boot is even further away, so little chance of that.

@Igor Hmm this will be a challenge. Are you sure you can't download the bundles, or is some other error? Send me a log file please? I'll find a way, if the bundle can't be downloaded, to revert to fetching from origin, so Google mirror can be used. I guess folks in China and others might need that as well. Also if I ever get around to the gitbundle caching storing in docker layer in GCHR.IO, we could have a workflow that publishes ready-to-go, 300mb gitbundles for each version (5.15 / 5.17 etc) and solve this once and for all. I've thought and studied a lot about this problem, and there' s another possible, although very different, solution: using a single, shared `.git` tree with all remotes and all branches, huge, possibly seeded by bundle, for all kernels, and then for each built kernel, a separate working copy. See https://git-scm.com/docs/git-worktree -- this would be even more complex to manage than already, but would be more efficient when building say more than 2/3 kernels, and much more efficient when building many different versions (eg 4.19 / 5.15 / 5.17). But that is maybe for armbian-next-next. Thanks, will investigate, I am still building in Impish and is somehow working... Those are very interesting. I will investigate. I've small patches to make compatible with newer gcc for some other families, like xu4, so I have hope. Ooops, this was not supposed to be included in armbian-next, was trying to reproduce some boards that used to work 1/2 years ago but no longer boot. I do think BRANCH=classic, generally 5.10.y, is a good idea, but armbian-next is not the place for this. Yeah. I've too many manual merges and at a certain point I lost track. The fact that our docker support uses a "docker" config file that itself re-launches the build inside docker is very confusing. The template-copying style of this is also hard to maintain... At a certain point I thought to rewrite, never did, and merges were sloppy. I will review. My bad. Regarding kernel-drivers.sh, this is one of my main to-do items. I've good progress on 'fasthash' (which is required for kernel-drivers) so this is getting close to doable. Will keep you posted. @balbes150 thanks for your tests and posting the logs. These errors were a bit unrelated and came from master, "nala" package which was added by "desktop" sources, and desktop sources were included in CLI builds in master. Dunno if really intended... Since I rebase irregularly sometimes this was fixed fast in master but -next lagged quite a bit. I've disabled this in armbian-next (sources only added for desktop builds again) but will lead to discussions, since it was already accepted into master, and armbian-next already breaks a lot of other stuff. Those kind of changes are marked *breaking change* in the commits in armbian-next when I can identify them. There's quite a few. ------------------ To all here: thanks a lot for the tests and reporting the problems. I have already started a huge rebase against master, fixing a few things and doing a few CI runs to catch the more obvious problems. Also rebasing showed frozen kernel tags, armbian-next has all that is needed to do that externally now, so also working some long overdue features. Will post when it's pushed!

People, I feel bad about hijacking this thread with armbian-next stuff. Could someone with forum-fu move armbian-next related posts to its own thread? Maybe in a contributor-only forum?

Thanks, a pesky short-circuit bug was leftover in sunxi_common at the end of family_tweaks_bsp. Fixed. Please pull and try again... Will be possible (and default) soon. Real Logs are now produced inside the WORKDIR (in .tmp) in text + ANSI colors format, and during cleanup, processed and aggregated into a single file HTML. This has brought joy to some and hatred to others. Sorry to offend your sensibilities... But do expand on "don't play nice with my tools", which tools? We wanna play nice. Yes. `PRESERVE_WORKDIR=yes` won't cleanup the workdir in .tmp -- this leaves over a gigabyte of stuff but should be safe. Also `PRESERVE_SDCARD_MOUNT=yes` will preserve $SDCARD, $MOUNT (and thus $LOOP) -- take much care in this case, things are left mounted and you can easily shoot yourself in the foot (eg, destroy your host) if use this and is not careful. Other interesting things you might like - `SHOW_LOGS=yes` shows the output of run commands - `SHOW_COMMAND=yes` shows the invocation of each said command Thanks for trying and reporting!

This is deviating from the main subject, but just to clarify a bit what armbian-next is trying to do with kernel's git: - First: it knows the main version (eg: 5.15, 5.16) of every kernel to be built, even if it's not from mainline sources, for example vendor source, say a legacy. Idea was taken from sunxi; I've modified every family to have that information (similar, but not identical, to KERNELBRANCH) - When fetching kernel sources, say for 5.15, it will: - check for 5.15 git bundle "warm" cache. if found, fetch from it and skip down to item X. if not, continue. (point of interest "1".) - check for Torvald's git "cold" cache. if found, fetch from it. if not, download it via HTTP+CDN from kernel.org's non-git mirrors, then fetch from it. this is slow, error prone, and many gigabytes; it has all "master" history, but not stable branch history. - "X": add a 2nd remote which is stable kernel git from whatever mirror we want. fetch the latest 5.15.y from it, and all stable tags too. this is always done even if non-mainline kernel. - if warm bundle for 5.15 does not exist yet: export it, in the process making it shallow before 5.15-rc1. include tags. (produced warm bundle is around 300mb and super fast to fetch from) - (point of interest "2") - (if non-mainline kernel) add a 3rd remote "origin" and fetch from it, just like we did before. this fetch is now in order of few megabytes or even a few kilobytes. With this we end up with something that does 0 shallow fetches from remotes (which is what kernel.org wants from us) but in the end is both shallow and contains tags that are useful for maintainers. The main improvement needed here are "point of interest" 1 and 2: those could try to offer "ready-to-go", around 300mb, shallowed-with-tags files that could be cached and offered to users just like we do for rootfs caches. So with 300mb we end up with a very usable git tree instead of a tarball of just-source for half of that. The fact that the bundles age (and are out of date) is irrelevant, since we always fetch (only differences, never shallow) from remotes, so it's easy to update. I am investigating using freely available resources (eg: OCI registries on Github) to store and fetch those "gitbundle caches" so that people don't have to suffer the 4gb marathon that is fetching from torvald's bundle.

Here's some info on armbian-next status: - In development since October/2021 (thus 7 months) - Currently sitting at 189 commits, almost all lib/**/*.sh files are changed. - More than 17 rounds of manual merges performed manually against master. Sometimes we have 1-2 weeks of delay, but it's mostly master-equivalent. Each round takes 2-5 hours of carefully reviewing every commit that landed on master and maybe rewriting it. - Huge changes around logging and error control. Errors now stop the build. Logs contain actionable information. - Rewritten kernel building and packaging. There's no more packaging patches or builddeb hacks. Kernel-headers support cross compiles across all arches. No more byteshift patch. Still needs tuning. - Completely changed git handling, specially for kernel. This still needs handling of previously git://, now http:// bundle downloading as reported above, and caching. - Patch hashing is half-rewritten, still needing work. Now same code can both actually patch and produce hashes, making it consistent. "fasthash" - apt-cacher-ng configuration for local usecase is now under Armbian control. - 100% working without any external/downloadable toolchains. - full support for building any arch under any other arch, including "reverse cross-compiles" and running of rkbin/fip x86-only binaries. - source code is now properly formatted according to shellfmt, and 90% of shellcheck warnings are squashed. A lot remains still. - 20x faster kernel rebuilds, 2-3x faster full builds due to consolidated make invocations, benefitting from bintree caching as well as ccache. - most compression points changed to zstd, yielding multicore (de-)compression and thus much faster image building during cache hits. - full support for DKMS modules, eg nvidia-drivers and ZFS, under any cross compile scenario. - data channel for metadata communication, config extraction, matrix generation etc by having stdout clean for machine data. logs to stderr. - per-build TMPDIR affecting and protecting all `mktemp` invocations - all traps rewritten under a cleanup/trap manager. - full HTML-based, single-file, full build log artifact. no more output/debug. - many others I forget. All that said, I've still a ton of stuff pending before this can 100% replace current master: - EXTRAWIFI-style manual patching of things is not idempotent nor proven working with "fashhash" and probably causes full kernel recompiles, negating previous benefits until fixed. - (server-based) Caching of warm/cold git bundles, to avoid people downloading 4gb+ of git sources. - kernel-headers will probably be unified with kernel-sources, since they contain mostly the same stuff. - extra-buildpkgs is not touched nor run, and safe to assume needs lots of work. - certain CI-only build ops like "rebuild roofs only" and "build all BSPs" and "build all u-boots" never run so probably won't work. Some few brave souls like @NicoD, @Rich Neese, and @going have actually tried armbian-next and every single try there's stuff to fix, thanks for trying and reporting. TL,DR: armbian-next is doing well, thanks. Work continues. It might not be ready for merge, will it ever be? I will keep it rebased as much as possible. There's no pressure. Try it out, and report. Thanks!

sorry, no. I can't really say or fix anything about 21.08.x at this time. It's just too old of a codebase, and using vendor uboot, so with no future. Personally I boot the N2+ from SPI to USB3 pendrive (not cabled!) and have decent experience with 22.02, but can't say much about the eMMC + SD combo. 22.02 includes mainline uboot and kernel fixes for N2 which are essential to any sanity, but might still have some rough edges. Try the RC's! If you had working 21.08 at any point you're just lucky: I never had a stable N2 on 21.08, literally problems everywhere. Good luck!

Hello, I guess you've updated the Kernel but not the bootloader? The PR you mention is for Kernel, but there is a related u-boot PR that goes hand in hand. Tread carefully, this is all being done without any support from HardKernel and things may break. Suggestion is to try a nightly build on a separate SD card, that will have matching bootloader and kernel, and determine if that works OK before updating your running system.

Oooh, it seems there's updated images from 23/Jan up at https://www.armbian.com/odroid-n2/ - but not sure what branch they're built from.

Yeah, valid to mention I did a big rework of current/5.10.y last year that has not been included in any release yet. The images we have up (from August?) do not include them and are very problematic. The rework fixes a ton of problems and usually brings 5.10.y/current stability, apart from reducing the impact of N2 on the other G12's. In the process, UHS speeds for SD were lost (they were implemented via vendor uboot 2015 hacks by HK), governor was set to performance.... all things that may have been fixed in latest versions of mainline u-boot; and Neil Armstrong has already pointed me to fixes for a bunch of things, so only more good things to come. Some users have built from master or tried my unofficial images and feedback was pretty good. If only HK would support us somehow or at least work in a more mainline-oriented way on their own branches.... For next release I hope we can keep 5.10.y as current for some real users to try and after that I'll rework edge kernel to bring 5.15 / 5.16 goodness, uboot 2022.01 already is done by Igor, 100% mainline even for SPI boot (and even SATA boot on HC4). Right now my N2 is "in production" and has 55 days uptime with 5.10.83-meson64 and is a beast; I'd accept a second one for development, although I'm focused on the HC4 now.

Please me @rpardini

Yes, but fear not, _fragments_ lives on in my branches. I will try to upstream it again eventually, maybe together with more useful features/refactoring based on it.

@Jeremiah Cornelius official RC builds (21.08_1) are up at https://www.armbian.com/khadas-vim3/

Yeah, don't bring boards on vacation is a decent policy, though often violated I hear. I realized after posting that we're on a tight schedule (Armbian 21.08 release) so I jumped the gun and sent a very basic PR, but is 100% untested. It's Armbian's default meson64 kernel, mainline u-boot 2021.04 with vim3 default config, and the a311d DTB from mainline kernel, and that's it. It could benefit from your work (asound file, any gpu/X11 settings, etc) but should otherwise be very similar to the N2 image I understand you based off. If it boots, it's a win already. Enjoy your vacations!

Hey @Jeremiah Cornelius great stuff. I've been working on removing the ODROID-specific cruft from the G12B family in armbian/build. That is almost done now (some damned patches are still in the way) so I could start work on adding VIM3 officially. Are you on Armbian's Discord? Catch me there in #amlogic as 'rpardini' We could surely benefit from each other's work (and I from the fact you have a VIM3 to test on ;-) Let me know

Hey @Jeremiah Cornelius I've been working on something similar it seems, thanks to @lanefu for making the connection. Although the path I took was directly modifying the Armbian build scripts, instead of modifying a live image and then re-packing it. My fork of Armbian is quite complex, but I'm slowly splitting and reorganising things (and sometimes writing a metaprogramming Bash framework in the process ) A few points that may be of interest to you: - Onboarding replaced by cloud-init, instead of any interactive onboarding, I configure cloud-init to read some YAML files from /boot. /boot/network-config can configure all aspects of networking (including wifi), /boot/meta-data is just that, and /boot/user-data can do almost anything, create users, passwords, import SSH keys, install packages, run scripts, resize rootfs, create SSH host keys, etc. cloud-init is of course focused on cloud images, but actually can be used on any environment, if you can stomach its YAML and hammer on its config. With cloud-init comes netplan.io (in place of NetworkManager) and some other stuff that may not be appropriate for your use-case so take with a grain of salt. Also my implementation is not bullet-proof yet, but it aims to work just like the Ubuntu RaspberryPi 4 "Server" image (also has cloud-init out of the box). Most of it is done in https://github.com/rpardini/armbian-build/blob/rpardini-fragments/userpatches/fragments/cloud-init/cloud-init.sh (including the .not_logged_in_yet thing) Some other related stuff in https://github.com/rpardini/armbian-build/blob/rpardini-fragments/userpatches/fragments/more_like_ubuntu_cloud.sh (which disables a bunch of stuff in the BSP, during BSP-generation, so that reinstalls/upgrades of the bsp package do not ruin everything). - A trick I call rootfs-in-rootfs, during the image build process, I capture an e2image ext4 dump of the produced ext4 rootfs and then include it in the rootfs. A bit like having a directory full of files, creating a .zip of it, then including that .zip file inside itself but without the compression. Then boot the SD card (which includes the rootfs dump inside) and run "e2image -racp /root/rootfs.ext4.e2img /dev/sda2" for example. The nice thing is that e2image (part of e2fsprogs) can do a "smart" copy (-c), knowing that reading is cheaper than writing, it can read both the source and destination and only write the changed blocks to the target. This allows for a very, very fast version of "nand_sata_install", especially on the 2nd+ "flash". More speed comes from reading/writing some hundred-thousand whole blocks instead of possibly millions of files that rsync would need to inspect individually. The relevant code fragment is https://github.com/rpardini/armbian-build/blob/rpardini-fragments/userpatches/fragments/rootfs_e2img_inside_rootfs.sh Hopefully these will be of some help, Ricardo