Weekly developers meetup

I'm in a minority time zone, so will often not be able to attend meetings that suit the majority.
I would like to read the minutes though, just to keep up with what's going on, and in case I can contribute.

I expect we will shuffle this time around a bit in the future or even split into two teams.

I promised that I would try to run a test build in the "armbian-next" branch.
The first run with my test config failed.

 

[💥] Error occurred in main shell [ code 2 at /home/leo/armbian/lib/functions/configuration/main-config.sh:59
       do_main_configuration() --> ./lib/functions/configuration/main-config.sh:59
prepare_and_config_main_build_single() --> ./lib/functions/main/config-prepare.sh:109
      cli_standard_build_run() --> ./lib/functions/cli/cli-build.sh:12
     armbian_cli_run_command() --> ./lib/functions/cli/utils-cli.sh:121
              cli_entrypoint() --> ./lib/functions/cli/entrypoint.sh:154
                        main() --> ./compile.sh:52
 ]
[🌿] Built ANSI log file [ /home/leo/armbian/output/logs/armbian-build-28afec36-cd02-43ba-be63-5334f3516cba.ansitxt.log ]
[💥] Error occurred in main shell [ code 2 at /home/leo/armbian/lib/functions/cli/utils-cli.sh:215
cli_standard_relaunch_docker_or_sudo() --> ./lib/functions/cli/utils-cli.sh:215
  cli_standard_build_pre_run() --> ./lib/functions/cli/cli-build.sh:5
 armbian_cli_pre_run_command() --> ./lib/functions/cli/utils-cli.sh:107
              cli_entrypoint() --> ./lib/functions/cli/entrypoint.sh:67
                        main() --> ./compile.sh:52
 ]

I very briefly looked at the code that the tracer output.

For lib/functions/configuration/main-config.sh:59 file

This is a very gross mistake. The name of the remote repository can be anything. "origin" is a common option, but far from the only one.

leo@vm-jammy:~/armbian$ echo "$(git remote)"
armbian
host

 

 

[🌱] Downloading Kernel bundle [ stable; this might take a long time ]
Initializing download: https://mirrors.edge.kernel.org/pub/scm/.bundles/pub/scm/linux/kernel/git/stable/linux/clone.bundle
File size: 4.2958 Gigabyte(s) (4612578906 bytes)

@rpardini We've already discussed this. And I see it again. Who should I bill for such a large traffic?

Recording

Slide deck for the armbian-next presentation

armbian-next - meeting 21.dec.2022.pdf

5 hours ago, going said:

This is a very gross mistake

 

True, and same gross mistake exists on `master` branch, but it silently fails and is ignored. armbian-next stops on any errors now, which you just hit a case.

Thanks for reporting

 

4 hours ago, going said:

Who should I bill for such a large traffic?

 

Definitely bill me! 😇

Meeting video recording can be watched: here (~120M, mobile friendly mp4 format)

I tried armbian-next to run a native build (on an ARM device), and got errors. Why is a bunch of unnecessary shit being put on ARM (snap, gcc-riscv, etc.)?

 

👉]    Reading package lists...
[👉]    Building dependency tree...
[👉]    Reading state information...
[👉]    Package ccze is not available, but is referred to by another package.
[👉]    This may mean that the package is missing, has been obsoleted, or
[👉]    is only available from another source
[👉]    Package python3-pip is not available, but is referred to by another package.
[👉]    This may mean that the package is missing, has been obsoleted, or
[👉]    is only available from another source
[👉]    Package distcc is not available, but is referred to by another package.
[👉]    This may mean that the package is missing, has been obsoleted, or
[👉]    is only available from another source
[👉]    No apt package "axel", but there is a snap with that name.
[👉]    Try "snap install axel"
[👉]    No apt package "tree", but there is a snap with that name.
[👉]    Try "snap install tree"
[👉]    E: Package 'distcc' has no installation candidate
[👉]    E: Package 'python3-pip' has no installation candidate
[👉]    E: Unable to locate package python2-dev
[👉]    E: Package 'ccze' has no installation candidate
[👉]    E: Unable to locate package colorized-logs
[👉]    E: Unable to locate package tree
[👉]    E: Unable to locate package pbzip2
[👉]    E: Unable to locate package axel
[👉]    E: Unable to locate package crossbuild-essential-armel
[👉]    E: Unable to locate package crossbuild-essential-amd64
[👉]    E: Unable to locate package gcc-riscv64-linux-gnu
[👉]    --> 🧽   78: 3365 - 3365 - 3365: hBE: trap: main_trap_handler [ ERR and 100 trap_manager_error_handled: short_stack:/home/user/armbian-build-next/lib/functions/logging/runners.sh:186 ]
[💥]     👆👆👆 Showing logfile above 👆👆👆 [ /home/user/armbian-build-next/.tmp/logs-599221b8-4094-41cf-af68-4d18190ab396/004.000.prepare_host.log ]
[💥] Error occurred in main shell [ code 100 at /home/user/armbian-build-next/lib/functions/logging/runners.sh:186
 run_host_command_logged_raw() --> ./lib/functions/logging/runners.sh:186
     run_host_command_logged() --> ./lib/functions/logging/runners.sh:169
                host_apt_get() --> ./lib/functions/logging/runners.sh:139
        host_apt_get_install() --> ./lib/functions/logging/runners.sh:132
  install_host_side_packages() --> ./lib/functions/host/host-utils.sh:60
   install_host_dependencies() --> ./lib/functions/host/prepare-host.sh:291
                prepare_host() --> ./lib/functions/host/prepare-host.sh:111
             do_with_logging() --> ./lib/functions/logging/logging.sh:112
   main_default_build_single() --> ./lib/functions/main/default-build.sh:27
      cli_standard_build_run() --> ./lib/functions/cli/cli-build.sh:16
     armbian_cli_run_command() --> ./lib/functions/cli/utils-cli.sh:121
              cli_entrypoint() --> ./lib/functions/cli/entrypoint.sh:154
                        main() --> ./compile.sh:52
 

 

I looked at the table by the link. I don't understand what practical use it is? "No description of mbox" - has no use. "rebase" - it is not clear why. the rest of the items are the same. does not do the main thing - does not check the error when using patches

https://rpardini.github.io/linux/armbian-next-media-6.1-20221220.html

On 12/25/2022 at 8:11 AM, balbes150 said:

native build (on an ARM device), and got errors

 

Thanks for trying.

These, of course, are not supposed to happen.

It seems there is a big problem with the host's sources.list, since it can find basic packages like "python3-pip" which should be available everywhere.

Could you please indicate what is it that you're building on, or post the complete log?

 

Ref "having a lot of shit": riscv64 is now a supported architecture, even on master, so it has been added to dependencies even before any armbian-next stuff.

Ref "snap" is related to above: it can find the proper packages so for some reason is saying shit about snap.

 

On 12/25/2022 at 8:15 AM, balbes150 said:

"No description of mbox" - has no use

 

It's pointing out you have a bare patch, not correctly mbox formatted. Those patches are missing: the Author, the Date, and the Subject, and more evidently, the Signed-off-by. So it's a cosmetic problem.

 

On 12/25/2022 at 8:15 AM, balbes150 said:

"rebase" - it is not clear why

 

Means that the patch only applied due to "patch (the utility)" being fuzzy and seeking for offsets. It would NOT apply with "git am" or similar strict tools. It's somewhat cosmetic, better solved early than later.

 

On 12/25/2022 at 8:15 AM, balbes150 said:

main thing - does not check the error when using patches

 

Curious about what you mean there. 

 

1 час назад, rpardini сказал:

It seems there is a big problem with the host's sources.list, since it can find basic packages like "python3-pip" which should be available everywhere.

Please note that these are packages that he is trying to install into the host system, i.e. these dependencies are written somewhere as mandatory for the host system where the build is launched. The assembly with GIT-Master works fine on the same host system, i.e. all the packages and dependencies necessary for the assembly system to work are already installed and the assembly works without problems. But the NEXT version is trying to install all this extra junk. You can easily reproduce it yourself. Take any ARM device (power and other parameters do not matter), run for example from SD\USB a ready-made Armbian Jammy image for this device and see what happens when you try to start building any image.

1 час назад, rpardini сказал:

It's pointing out you have a bare patch, not correctly mbox formatted. Those patches are missing: the Author, the Date, and the Subject, and more evidently, the Signed-off-by. So it's a cosmetic problem.

 

1 час назад, rpardini сказал:

Means that the patch only applied due to "patch (the utility)" being fuzzy and seeking for offsets. It would NOT apply with "git am" or similar strict tools. It's somewhat cosmetic, better solved early than later.

 

IMHo - all these checks and the "problems" found in patches have no practical use.

 

1 час назад, rpardini сказал:

Curious about what you mean there. 

 

A practical test of applying a patch to the source code, with a detailed indication of exactly which part of the patch code is not applied. You can see an example in the Libreelec build system. There is a detailed patch overlay log and you can immediately see which piece of patch code is problematic. This is especially true for large patches with a large number of fix points in different places. Now there is a patch overlay log in the build log, and there is only scanty information there, which makes it difficult to understand where the problem is in patches, especially if there are many changes in one file.

4 hours ago, balbes150 said:

But the NEXT version is trying to install all this extra junk

 

Again, all this "junk" is already on master. https://github.com/armbian/build/blob/master/lib/functions/host/prepare-host.sh#L53

The main difference: armbian-next removes the conditionals, thus installing this junk for everyone, even if not needed.

This was done on purpose so problems show up instead of being there hidden (exactly like happened)

I might reconsider this, specially after we've cleaned up the junk.

So thanks for feedback.

 

4 hours ago, balbes150 said:

Take any ARM device

 

I build regularly on arm64 with no problems, everyday, from scratch. If your host can't find "python3-pip" (https://packages.ubuntu.com/jammy/python3-pip), it has an unexpected apt / sources /etc.

For example, missing the "universe" repository or some such? 

Like you mentioned, this is "host" packages/sources.list, so the script shouldn't even try to fix it.

I'd be interested to know what is in your sources.list so I can try to detect the problem and alert instead of just breaking.

Again, thanks for the feedback, since if it breaks for you, probably will for others.

 

4 hours ago, balbes150 said:

IMHo - all these checks and the "problems" found in patches have no practical use.

 

Agree to disagree. Patches, like everything, need basic hygiene. But yeah I get your point.

 

4 hours ago, balbes150 said:

A practical test of applying a patch to the source code, with a detailed indication of exactly which part of the patch code is not applied. You can see an example in the Libreelec build system. There is a detailed patch overlay log and you can immediately see which piece of patch code is problematic. This is especially true for large patches with a large number of fix points in different places. Now there is a patch overlay log in the build log, and there is only scanty information there, which makes it difficult to understand where the problem is in patches, especially if there are many changes in one file.

 

Interesting. armbian-next handles that in a few different ways:

1) .patch files that contain multiple patches (in mbox format) are split, and applied individually, making it much easier to understand what broke, when it breaks.

2) patching "rejects" (hunks that failed to apply) are now shown in the log, so even if it's a large bare patch, with 10 hunks, and only 1 hunk fails, you can now find what broke more easily.

Of course showing those errors better in the logs is still TODO. (Enable SHOW_LOG=yes, maybe even SHOW_DEBUG=yes to actually see it)

 

*All that said...* there are patches that apply without errors and still make a terrible mess anyway. Look at the result of applying "board-pbp-add-dp-alt-mode.patch" (rockchip64) for example. It's completely wrong.

 

 

 

 

 

 

 

 

 

 

 

 

14 часов назад, rpardini сказал:

thus installing this junk for everyone, even if not needed.

Why install something that is not needed????

 

14 часов назад, rpardini сказал:

I build regularly on arm64 with no problems

The correct system check is to record a clean image and carry out all the steps strictly according to the initial instructions, on a completely clean \ new system.

 

 

On 12/29/2022 at 7:53 AM, balbes150 said:

Why install something that is not needed????

 

On 12/29/2022 at 7:53 AM, balbes150 said:

on a completely clean \ new system

 

Seems like there's cognitive dissonance here...

 

4 часа назад, rpardini сказал:

Seems like there's cognitive dissonance here...

do not confuse the general testing process on a clean system and the installation of any unnecessary junk that is not needed for the selected specific image (platform) into one pile.

On 12/25/2022 at 8:11 AM, balbes150 said:

E: Package 'python3-pip' has no installation candidate

 

On 12/30/2022 at 5:03 PM, balbes150 said:

do not confuse the general testing process on a clean system and the installation of any unnecessary junk

 

Do not confuse a "clean system" with a "system that is missing sources and can't install a basic package", vs "junk". The junk is/was already there, and should work first, before deciding we don't actually need it for one specific scenario/wishes.

 

This discussion is pointless, I just wanted for you to send me the sources.list so I can understand/detect/fix this problem. If you refuse, I still thank you...