rpardini
-
Posts
132 -
Joined
-
Last visited
Content Type
Forums
Store
Crowdfunding
Applications
Events
Raffles
Community Map
Posts posted by rpardini
-
-
On 5/12/2022 at 6:04 PM, rvalle said:
I also use cloud-init for server customization
Cool stuff, thanks for the explanation. I've similar setup, although I don't use root LVM, and cloud-init meta is in /boot ext4 or /boot/efi (vfat).
Partitioning is quite complex intermingled code between partitioning proper, calculating sizes, and setting up filesystems.
I've added hooks to be able to "override it all" but haven't had time to make it better composable, much less in bash.
But what I was asking about was the your original mention of "docker-shell" or something, for image build? kernel build? I've probably messed that up in -next.
-
-
3 hours ago, rvalle said:
I get this from docker-shell
Eh, no idea. Fragments of code handling docker are scattered all over the codebase, and haven't been properly handled yet in armbian-next.
(1-line explanation: armbian-next has error control enabled, and command invocations that previously failed and were ignored now make the build stop).
Would be great if you could explain the use case for what you're trying to do...
-
1 hour ago, rvalle said:
I just want to build the image with a patch of mine too. See if we are lucky and get it to work.
Should be doable, patch dirs are empty so just drop stuff in there.
1 hour ago, rvalle said:Are you using Tobetter code from Hardkernel? We have a discussion over there because most modules are disabled in the kernel even if they have nothing to do with the actual silicon. He is looking into it, be aware.
legacy == 4.19 from HK official repo.
edge == 5.18-rcX from tobetter's repo.
both with vendor u-boot
so all moving targets, tobetter has finally taken to rebasing his branch.
-
Yes. I don't think it's even near being merge-able right now, small bugs but are showstoppers for most people, and I can't handle them all by myself in such timeframe, including handling all the Jammy fixes and a dozen rockchip kernels, extlinux, nand-sata-install, etc.
And once merged a whole slew of things will show up (extras buildpkgs, prebuilt, rootfs caching, repo publishing, git bundle stuff, most of the GHA stuff that normal developers hardly ever run).
The concepts behind armbian-next are almost fully realized, but the fruits still have to be collected. Otherwise it seems pointless...
I will keep it rebased as much as possible, so the effort lives on, only without pressure.
Before merge, documentation has to be written too, so that will take a while.
work continues!
-
1 hour ago, rvalle said:
I am trying to make sense of your code.
Don't try too hard. Most things don't make sense. 😉
Find the board file, then the family, and understand the hooks that are in place and when they're called.
1 hour ago, rvalle said:The code is in your rpardini/extensions branch, but was it from armbian-build/master
rpardini/extensions is my work branch. It's a huge mess. Don't waste your time...
Everything you need for the M1 is already in armbian/build in branch 'armbian-next'.
The ODROID-M1 uboot code is very convoluted right now since I'm using vendor uboot's "make.sh" in addition to Armbian's own.
Also a custom partitioning scheme is required, also implemented in a hook in the board file.
It can _only_ be built on x86 right now due to vendor's use of amd64-only rkbins there.
So right now this is very not standard.
If/when this board stabilizes I'll work on making it less messy.
-
1 minute ago, rpardini said:
We'll have a hard time without python2 for those old uboots
Nah, Jammy has python2, it does not have python-is-python2 gimmick which just symlinks or update-alternatives.
I'll have to hack around this manually.
-
On 4/25/2022 at 6:38 PM, Igor said:
we should exit if HOST is less them Jammy, since Git is too old.
23 minutes ago, Igor said:If you start on Jammy,
https://github.com/armbian/build/blob/armbian-next/lib/functions/host/prepare-host.sh#L46is no more.
Ahn, ok, this is recent addition for 1 runner that was failing uboot, I thought you meant GIT version.
We'll have a hard time without python2 for those old uboots, so it's a symptom of shit to come.
But thanks for the catch, I'll fix that ASAP, and probably more when I move builders to Jammy.
27 minutes ago, Igor said:Also office unblocked at kernel.org
I still couldn't find time to work on this.
-
Ok some images are up for testing.
Both HKs' 4.19 and tobetter's 5.18-rcX were updated. Maybe something is fixed. Tobetter commits signal support for PCIe.
For those who are brave:
https://github.com/rpardini/armbian-release/releases/download/20220508d/Armbian_20220508d-rpardini_Odroidm1_jammy_legacy_4.19.219.img.zst (probably no HDMI)https://github.com/rpardini/armbian-release/releases/download/20220508d/Armbian_20220508d-rpardini_Odroidm1_jammy_edge_5.18.0.img.zst (probably no PCIe/SATA/USB3)EDIT: old images removed. Look for later posts for newer ones
unzstd them, burn to SD, insert SD, power on.
recommend to use UART
Have fun @Neo2SHYAlien @Stan Pak @N4IRS @Slartibartfast9536 @Igor
-
Ok, a weekend later, here's armbian-next v64.
https://github.com/armbian/build/commits/armbian-next - do `git pull --rebase` and build away and report problems here 😉
Builds are green. https://github.com/rpardini/armbian-release/actions/runs/2289642908
Images are up for download: https://github.com/rpardini/armbian-release/releases/tag/20220508d
(a few novelties there. eg, ODROID-M1)
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
8 hours ago, Igor said:I would not push on moving everything to distro compilers at once. What works, works and we do this gradually. Going on half now, removing them all in one year. Something like that. Even today, sometimes we will need to deal with a special compiler, so support has to remain. And a few of them if there is too much work with adopting the code.
No fun in that @Igor😉 -- I've fixed the Makefiles for cubie-i and clearfog u-boot so they build. Those were exactly the same problem as from xu4: 2017/2018 u-boot, that original Makefile wanted "armv5" target which does not exist anymore in newer gccs. https://github.com/armbian/build/commit/0a3fc81b5e519a11ac41c1d196b076a91ce38a1b
Most of those boards are actually armv7a, so I patched and they built. If they _work_, we'll only know if some of you download the above images and try it out (cubie, clearfog, odroidxu4 at least). Thanks!
Many other changes/fixes and a few breaking ones, like removal of LIB_TAG/.ignore_changes for @lanefu
I've not tested desktop building so here's when @Rich Neese comes in, let me know.
Thanks for all the tests!
-
I've some images, but they're really quite far from usable right now.
Vendor kernel 4.19 I can't get HDMI to work. Everything else works. I'm using it to build Armbian itself and works great with NVMe and the 8gb RAM.
Mainline 5.18-rcX has working HDMI, but no NaNeng PHY, so no PCIe/NVMe, no USB3, and no SATA. tobetter is working, also work in quartz64a and other rk3586/8 will benefit this.
nand-sata-install will not work with this unless I find a way to flash uboot to SPI (and this remove petitboot, but there is no official petitboot recovery from HK yet)
and mainline u-boot is even further away, so little chance of that.
-
On 4/25/2022 at 6:38 PM, Igor said:
I am still blocked
Can't use bundles from git.kernel.org ... USE_MAINLINE_GOOGLE_MIRROR="yes", what about other mirrors?
@Igor Hmm this will be a challenge. Are you sure you can't download the bundles, or is some other error? Send me a log file please?
I'll find a way, if the bundle can't be downloaded, to revert to fetching from origin, so Google mirror can be used. I guess folks in China and others might need that as well.
Also if I ever get around to the gitbundle caching storing in docker layer in GCHR.IO, we could have a workflow that publishes ready-to-go, 300mb gitbundles for each version (5.15 / 5.17 etc) and solve this once and for all.
I've thought and studied a lot about this problem, and there' s another possible, although very different, solution: using a single, shared `.git` tree with all remotes and all branches, huge, possibly seeded by bundle, for all kernels, and then for each built kernel, a separate working copy. See https://git-scm.com/docs/git-worktree -- this would be even more complex to manage than already, but would be more efficient when building say more than 2/3 kernels, and much more efficient when building many different versions (eg 4.19 / 5.15 / 5.17). But that is maybe for armbian-next-next.
On 4/25/2022 at 6:38 PM, Igor said:we should exit if HOST is less them Jammy, since Git is too old
Thanks, will investigate, I am still building in Impish and is somehow working...
On 4/25/2022 at 6:38 PM, Igor said:- Clearfog (current) fails at compiling u-boot (we can't go up with u-boot, too much work)
- Cubox fails at compiling u-boot (we can't go up with u-boot, too much work)
Those are very interesting. I will investigate. I've small patches to make compatible with newer gcc for some other families, like xu4, so I have hope.
On 4/25/2022 at 7:40 PM, Igor said:7th version of rockchip kernel
?
Ooops, this was not supposed to be included in armbian-next, was trying to reproduce some boards that used to work 1/2 years ago but no longer boot. I do think BRANCH=classic, generally 5.10.y, is a good idea, but armbian-next is not the place for this.
On 4/25/2022 at 7:40 PM, Igor said:config/templates/config-docker.conf is not merged well
lib/functions/compilation/patch/kernel-drivers.sh (behind / merged wrong , we have GITHUB as variable not hardcoded for those sources)
Yeah. I've too many manual merges and at a certain point I lost track. The fact that our docker support uses a "docker" config file that itself re-launches the build inside docker is very confusing. The template-copying style of this is also hard to maintain...
At a certain point I thought to rewrite, never did, and merges were sloppy. I will review. My bad.
Regarding kernel-drivers.sh, this is one of my main to-do items.
I've good progress on 'fasthash' (which is required for kernel-drivers) so this is getting close to doable. Will keep you posted.
On 4/26/2022 at 11:52 AM, balbes150 said:Trying to use the NEXT version causes a lot of errors. I have checked different combinations of images and settings (with disabling all package caching), but all options end in errors.
@balbes150 thanks for your tests and posting the logs. These errors were a bit unrelated and came from master, "nala" package which was added by "desktop" sources, and desktop sources were included in CLI builds in master. Dunno if really intended...
Since I rebase irregularly sometimes this was fixed fast in master but -next lagged quite a bit.
I've disabled this in armbian-next (sources only added for desktop builds again) but will lead to discussions, since it was already accepted into master, and armbian-next already breaks a lot of other stuff.
Those kind of changes are marked *breaking change* in the commits in armbian-next when I can identify them. There's quite a few.
------------------
To all here: thanks a lot for the tests and reporting the problems.
I have already started a huge rebase against master, fixing a few things and doing a few CI runs to catch the more obvious problems.
Also rebasing showed frozen kernel tags, armbian-next has all that is needed to do that externally now, so also working some long overdue features.
Will post when it's pushed!
-
People, I feel bad about hijacking this thread with armbian-next stuff. Could someone with forum-fu move armbian-next related posts to its own thread? Maybe in a contributor-only forum?
-
9 hours ago, schwar3kat said:
Orangepi-r1 (H2) build system failed near the end
Thanks, a pesky short-circuit bug was leftover in sunxi_common at the end of family_tweaks_bsp. Fixed. Please pull and try again...
9 hours ago, schwar3kat said:Can I switch off the time wasting and annoying markdown / html logging behaviour?
I don't want caterpillars, butterflies ants, bunches of leaves and comic explosions that don't make anything clearer and are unnecessary annoyances that don't play nicely with my tools.Will be possible (and default) soon. Real Logs are now produced inside the WORKDIR (in .tmp) in text + ANSI colors format, and during cleanup, processed and aggregated into a single file HTML.
This has brought joy to some and hatred to others. Sorry to offend your sensibilities...
But do expand on "don't play nice with my tools", which tools? We wanna play nice.
9 hours ago, schwar3kat said:Is it possible to keep the split .tmp files on error / build fail?
Yes. `PRESERVE_WORKDIR=yes` won't cleanup the workdir in .tmp -- this leaves over a gigabyte of stuff but should be safe.
Also `PRESERVE_SDCARD_MOUNT=yes` will preserve $SDCARD, $MOUNT (and thus $LOOP) -- take much care in this case, things are left mounted and you can easily shoot yourself in the foot (eg, destroy your host) if use this and is not careful.
Other interesting things you might like
- `SHOW_LOGS=yes` shows the output of run commands
- `SHOW_COMMAND=yes` shows the invocation of each said command
Thanks for trying and reporting!
-
This is deviating from the main subject, but just to clarify a bit what armbian-next is trying to do with kernel's git:
- First: it knows the main version (eg: 5.15, 5.16) of every kernel to be built, even if it's not from mainline sources, for example vendor source, say a legacy. Idea was taken from sunxi; I've modified every family to have that information (similar, but not identical, to KERNELBRANCH)
- When fetching kernel sources, say for 5.15, it will:
- check for 5.15 git bundle "warm" cache. if found, fetch from it and skip down to item X. if not, continue. (point of interest "1".)
- check for Torvald's git "cold" cache. if found, fetch from it. if not, download it via HTTP+CDN from kernel.org's non-git mirrors, then fetch from it. this is slow, error prone, and many gigabytes; it has all "master" history, but not stable branch history.
- "X": add a 2nd remote which is stable kernel git from whatever mirror we want. fetch the latest 5.15.y from it, and all stable tags too. this is always done even if non-mainline kernel.
- if warm bundle for 5.15 does not exist yet: export it, in the process making it shallow before 5.15-rc1. include tags. (produced warm bundle is around 300mb and super fast to fetch from) - (point of interest "2")
- (if non-mainline kernel) add a 3rd remote "origin" and fetch from it, just like we did before. this fetch is now in order of few megabytes or even a few kilobytes.
With this we end up with something that does 0 shallow fetches from remotes (which is what kernel.org wants from us) but in the end is both shallow and contains tags that are useful for maintainers.
The main improvement needed here are "point of interest" 1 and 2: those could try to offer "ready-to-go", around 300mb, shallowed-with-tags files that could be cached and offered to users just like we do for rootfs caches. So with 300mb we end up with a very usable git tree instead of a tarball of just-source for half of that.
The fact that the bundles age (and are out of date) is irrelevant, since we always fetch (only differences, never shallow) from remotes, so it's easy to update.
I am investigating using freely available resources (eg: OCI registries on Github) to store and fetch those "gitbundle caches" so that people don't have to suffer the 4gb marathon that is fetching from torvald's bundle.
-
Here's some info on armbian-next status:
- In development since October/2021 (thus 7 months)
- Currently sitting at 189 commits, almost all lib/**/*.sh files are changed.
- More than 17 rounds of manual merges performed manually against master. Sometimes we have 1-2 weeks of delay, but it's mostly master-equivalent. Each round takes 2-5 hours of carefully reviewing every commit that landed on master and maybe rewriting it.
- Huge changes around logging and error control. Errors now stop the build. Logs contain actionable information.
- Rewritten kernel building and packaging. There's no more packaging patches or builddeb hacks. Kernel-headers support cross compiles across all arches. No more byteshift patch. Still needs tuning.
- Completely changed git handling, specially for kernel. This still needs handling of previously git://, now http:// bundle downloading as reported above, and caching.
- Patch hashing is half-rewritten, still needing work. Now same code can both actually patch and produce hashes, making it consistent. "fasthash"
- apt-cacher-ng configuration for local usecase is now under Armbian control.
- 100% working without any external/downloadable toolchains.
- full support for building any arch under any other arch, including "reverse cross-compiles" and running of rkbin/fip x86-only binaries.
- source code is now properly formatted according to shellfmt, and 90% of shellcheck warnings are squashed. A lot remains still.
- 20x faster kernel rebuilds, 2-3x faster full builds due to consolidated make invocations, benefitting from bintree caching as well as ccache.
- most compression points changed to zstd, yielding multicore (de-)compression and thus much faster image building during cache hits.
- full support for DKMS modules, eg nvidia-drivers and ZFS, under any cross compile scenario.
- data channel for metadata communication, config extraction, matrix generation etc by having stdout clean for machine data. logs to stderr.
- per-build TMPDIR affecting and protecting all `mktemp` invocations
- all traps rewritten under a cleanup/trap manager.
- full HTML-based, single-file, full build log artifact. no more output/debug.
- many others I forget.All that said, I've still a ton of stuff pending before this can 100% replace current master:
- EXTRAWIFI-style manual patching of things is not idempotent nor proven working with "fashhash" and probably causes full kernel recompiles, negating previous benefits until fixed.
- (server-based) Caching of warm/cold git bundles, to avoid people downloading 4gb+ of git sources.
- kernel-headers will probably be unified with kernel-sources, since they contain mostly the same stuff.
- extra-buildpkgs is not touched nor run, and safe to assume needs lots of work.
- certain CI-only build ops like "rebuild roofs only" and "build all BSPs" and "build all u-boots" never run so probably won't work.
Some few brave souls like @NicoD, @Rich Neese, and @going have actually tried armbian-next and every single try there's stuff to fix, thanks for trying and reporting.TL,DR: armbian-next is doing well, thanks. Work continues. It might not be ready for merge, will it ever be? I will keep it rebased as much as possible. There's no pressure. Try it out, and report. Thanks!
-
sorry, no. I can't really say or fix anything about 21.08.x at this time. It's just too old of a codebase, and using vendor uboot, so with no future.
Personally I boot the N2+ from SPI to USB3 pendrive (not cabled!) and have decent experience with 22.02, but can't say much about the eMMC + SD combo.
22.02 includes mainline uboot and kernel fixes for N2 which are essential to any sanity, but might still have some rough edges. Try the RC's!
If you had working 21.08 at any point you're just lucky: I never had a stable N2 on 21.08, literally problems everywhere.
Good luck!
-
Hello, I guess you've updated the Kernel but not the bootloader?
The PR you mention is for Kernel, but there is a related u-boot PR that goes hand in hand.
Tread carefully, this is all being done without any support from HardKernel and things may break.
Suggestion is to try a nightly build on a separate SD card, that will have matching bootloader and kernel, and determine if that works OK before updating your running system.
-
13 minutes ago, rpardini said:
I did a big rework of current/5.10.y last year that has not been included in any release yet.
Oooh, it seems there's updated images from 23/Jan up at https://www.armbian.com/odroid-n2/ - but not sure what branch they're built from.
-
On 1/25/2022 at 2:28 PM, lanefu said:
any additional context to add about our N2 woes?
Yeah, valid to mention I did a big rework of current/5.10.y last year that has not been included in any release yet. The images we have up (from August?) do not include them and are very problematic.
The rework fixes a ton of problems and usually brings 5.10.y/current stability, apart from reducing the impact of N2 on the other G12's.
In the process, UHS speeds for SD were lost (they were implemented via vendor uboot 2015 hacks by HK), governor was set to performance.... all things that may have been fixed in latest versions of mainline u-boot; and Neil Armstrong has already pointed me to fixes for a bunch of things, so only more good things to come.
Some users have built from master or tried my unofficial images and feedback was pretty good. If only HK would support us somehow or at least work in a more mainline-oriented way on their own branches....
For next release I hope we can keep 5.10.y as current for some real users to try and after that I'll rework edge kernel to bring 5.15 / 5.16 goodness, uboot 2022.01 already is done by Igor, 100% mainline even for SPI boot (and even SATA boot on HC4).
Right now my N2 is "in production" and has 55 days uptime with 5.10.83-meson64 and is a beast; I'd accept a second one for development, although I'm focused on the HC4 now.
-
-
On 7/14/2021 at 8:54 PM, TRS-80 said:
I guess I felt like that great discussion in rpardini's PR was sort of shut down prematurely.
Yes, but fear not, _fragments_ lives on in my branches. I will try to upstream it again eventually, maybe together with more useful features/refactoring based on it.
-
@Jeremiah Cornelius official RC builds (21.08_1) are up at https://www.armbian.com/khadas-vim3/
-
Yeah, don't bring boards on vacation is a decent policy, though often violated I hear.
I realized after posting that we're on a tight schedule (Armbian 21.08 release) so I jumped the gun and sent a very basic PR, but is 100% untested.
It's Armbian's default meson64 kernel, mainline u-boot 2021.04 with vim3 default config, and the a311d DTB from mainline kernel, and that's it.
It could benefit from your work (asound file, any gpu/X11 settings, etc) but should otherwise be very similar to the N2 image I understand you based off.
If it boots, it's a win already.
Enjoy your vacations!
Asus thinker board 2s
in Framework and userspace feature requests
Posted · Edited by rpardini
wrong image link
@Excel I've an image. Unofficial. Unsupported. unzstd it, burn it to SD, put jumper J3 in Maskrom mode, UART at 1500000.
https://github.com/rpardini/armbian-release/releases/download/20220522c/Armbian_20220522c-rpardini_Tinkerboard_jammy_edge_5.17.9.img.zst(WRONG! That is for the TB1!)UPDATE: https://github.com/rpardini/armbian-release/releases/download/20220522c/Armbian_20220522c-rpardini_Tinkerboard-2_jammy_edge_5.17.9.img.zst (correct for the TB2)
Good luck!
Sorry boss, I've all these images laying around and can't help myself. 😉