Architecture for adding additional desktop enviromnents, and application groups

[Myy]

Greetings,

 

I'm currently modifying the Armbian build system, making it easier to add new desktop environments and application groups.

 

This architecture is subject to heavy modifications until it is merged with the current Armbian build system.

 

A repository with the modified build system is available here : https://github.com/Miouyouyou/armbian-desktop

 

Currently, I only tested adding Ubuntu Focal + XFCE and application groups, since I had to make sure that

the postinst and create_desktop_package.sh were actually aggregated correctly, now that they're split into multiple files,

and, for comparison, we only have such scripts for XFCE.

 

Also :

• the current list of packages for XFCE include unrelated packages (like neverball), added to test the packages files aggregation;
• there's a lot of debug messages added here and there, in order to have an overview of the current build state.

 

The actual packages list for other desktop environments and distributions, along with the scripts, still have to be added. (They're currently empty)

 

That said, I'd like to have a few maintainers test (quickly) the new architecture, in order to check if there isn't any obvious feature missing.

 

 

So, here's how to add a desktop environment with the new architecture :

 

Adding a desktop environment

 

Currently, only official repositories are supported.

 

Let's say that you want to add that new desktop environment "superduperde", that is now available on official on Debian/Ubuntu repositories.

First, focus on one specific distribution like focal (Ubuntu) or buster (Debian). In our example, will take focal.

We'll create our first configuration 'full', which should provide the DE along with all its specific apps, widgets and the kitchen sink.

 

• Create the directory config/desktop/focal/environments/superduperde/config_full
• Create the file config/desktop/focal/environments/superduperde/config_full/packages
• Open the packages file, add the list of packages for apt.

 

Then select it in the configuration menu, or pass the following variables to ./compile.sh :

BUILD_DESKTOP="yes" RELEASE="focal" DESKTOP_ENVIRONMENT="superduperde" DESKTOP_ENVIRONMENT_CONFIG_NAME="config_full"

Then test the resulting image !

 

Adding an appgroup

 

Appgroups are roughly the same. Let's say that you want to add remotecontrol to the appgroups for Ubuntu focal :

 

• Create the directory config/desktop/focal/appgroups/remotecontrol
• Create the file config/desktop/focal/appgroups/remotecontrol/packages
• Open the packages file, add the list of packages for apt.

 

Tale a look at https://github.com/Miouyouyou/armbian-desktop#the-architecture-added-to-the-build-system, for a more detailed documentation of the new architecture, and how to add special packages files for specific boards, and desktop configurations, along with specific debian/postinst and armbian/create_desktop_packages.sh files.

 

 

What needs to be quickly tested :

 

• Modifying an existing desktop environment (gnome, kde, ...).
• Adding a new desktop environment or window manager (mate-desktop, i3, fluxbox, ...)
• Adding a new appgroup
• Bonus points for adding a desktop environment with Wayland support (but remember that default installations come with terrible OpenGL support on most boards !)

 

 

 

[Igor]

Welcome to start testing desktop build process. It sits at our main repository, branch "desktop".

https://github.com/armbian/build

 

LIB_TAG="desktop"

 

[balbes150]

Results of a quick test of the native build using the Desktop version. For tests, I tried to build several options - XFCE \ MATE \ KDE in the "minimum" and "maximum" configuration for Station P1\M1 with the current and legacy core. The "minimal" configuration is an option where no group is selected on the last package selection screen. "Maximum" configuration-all package groups are marked. The build (native) runs without errors and the output is working images. Offer. Add the "standard" options in the settings (the necessary packages are selected automatically without additional user steps), this option is equal in the set of installed packages to the current position (the set of packages is equal to those that are currently installed in DE XFCE master branch). The "maximum" is automatically marked for all the available this DE packages (options). Custom setting ("minimum") when entering it, as now, no packages are marked, the user can, not specify anything, then the output will be the "minimum" version of DE or the user chooses the necessary groups\packages.

[balbes150]

I understand correctly, the new system (the desktop branch when building rootfs) uses the same set of packages to get the basic part, the difference is only in the composition of packages\scripts that are used for a set of packages that are described in variables 

 

PACKAGE_LIST_DESKTOP

 

in the master branch   ?

[balbes150]

I tried adding a new configuration to the build system that is (if possible) close to the current set of master branch packages when building a default image with DE XFCE. I checked the native build with this configuration option, the build passes without errors and the images work on P1\M1. If someone has time, please look and check the received configuration, maybe I missed some necessary packages (or added extra ones). Links to package lists for focal and buster.

 

https://github.com/150balbes/build/tree/armbian-tv-next/config/desktop/buster/environments/xfce/config_default

 

https://github.com/150balbes/build/tree/armbian-tv-next/config/desktop/focal/environments/xfce/config_default

 

I plan to send the PR after the other interested developers confirm that the configuration is correct.

[balbes150]

Option for "soft" integration into the main build system.

1. For each distribution, a default configuration is created, the composition of packages is equal to those used in the current version of the MASTER branch for official versions.

2. Each created set should be independent (references to "neighboring" distributions should not be used). This will allow  to accurately track the required set of packages, which applies only to the selected distribution (this is analogous to how it is now written in the script "configure.sh", all special packages for the distribution in its group  https://github.com/150balbes/build/blob/master/lib/configuration.sh#L202   ). And if necessary, it is easy to modify the set of packages strictly for the selected distribution, without breaking the assembly of other versions.

 

 

[balbes150]

fix bionic bullseye sid hirsute

 

https://github.com/150balbes/build/tree/armbian-tv-next/config/desktop/bionic/environments/xfce/config_default

 

https://github.com/150balbes/build/tree/armbian-tv-next/config/desktop/bullseye/environments/xfce/config_default

 

https://github.com/150balbes/build/tree/armbian-tv-next/config/desktop/sid/environments/xfce/config_default

[Igor]

Logic is now a bit different -> base desktop doesn't come with email client, chat client or browser ... but that is inside appgroup. Some appgroups like browsers, chat and perhaps another (not dermined yet) will be enabled by defalt. This will resoult in the same end package base.

[balbes150]

If  follow this logic, the current set of additional groups should correspond (when selecting all groups) to the current official set of packages for DE XFCE, which is currently used by the master branch. Otherwise, there is ambiguity. Some of the package groups must be selected to get a match, but other package groups will be "optional" (there are none in the current DE XFCE).

[Igor]

On 1/11/2021 at 8:26 AM, balbes150 said:

to the current official set of packages for DE XFCE

 

You think we should keep the exact base as now? Which packages are actually missing and what was added and its not needed? We were focused mainly into framework - this is not a final version. Package list is still an open topic. Especially for app groups. For base config there is little room for changes - it just should be minimal.

 

On 1/11/2021 at 8:26 AM, balbes150 said:

Some of the package groups must be selected to get a match

Which ones should be on by default? Should we leave this as per board? Like (almost) everything for desktop replacements or most powerful boards such as N2+, Pinebook Pro, Station, ... ?

BTW.
We are getting 1st draft of docs https://github.com/armbian/documentation/pull/125/files and we have a show stopper which needs to be addressed properly. If we don't fix this in time, merge to the master will be postponed. I wouldn't rush with this merge for all costs. Beta builds can already be published from "desktop", to catch more bugs.

[balbes150]

1 минуту назад, Igor сказал:

You think we should keep the exact base as now?

There is no need to save the current set of packages, I just brought them to the general state. So that the set in the new DESKTOP branch is (if possible) equal to the current state in the official versions. It's like a starting point for a merge.

 

7 минут назад, Igor сказал:

For base config there is little room for changes - it just should be minimal.

Perhaps then it is worth having a minimal configuration (set of packages) for DE ? That is, the minimum configuration describes (includes in the assembly) only the components that are necessary to form the  DE ? Move all other additional application packages to groups ? But then need to decide (discuss) which applications should be included in the basic delivery of the official version of DE XFCE, so that this set is the same for all official versions of Ubuntu\Debian with an officially supported DE (XFCE). this is necessary in order to form caches of package sets, by analogy, as it is now for DE XFCE for the MASTER branch.

 

17 минут назад, Igor сказал:

Which ones should be on by default?

Perhaps to simplify control (so as not to check which mandatory groups are included when building the official version of DE) it is easier to have a fixed configuration that is selected automatically when building official versions, for example config_armbian (with a description of all packages and applications in one place\file)?

[balbes150]

Question. Is it possible to have different package settings for different architectures aarch64\arm64 (64 bits) and separately for armhf\arm (32 bits) ?

[balbes150]

Strange behavior of the CLI image build. When natively building an image for p1 (with any kernel, desktop branch), everything goes without errors and I get CLI images (sid bullseye focal), but when trying to build on x86, I get this error for sid bullseye focal, for hirsute the build goes without an error. I need to understand if this is my local problem (my build environment on x86) or if there is such a problem.

 

 

 

(Reading database ... 37429 files and directories currently installed.)
Preparing to unpack .../armbian-config_21.02.0-trunk_all.deb ...
Unpacking armbian-config (21.02.0-trunk) ...
Setting up armbian-config (21.02.0-trunk) ...
Selecting previously unselected package zsh-common.
(Reading database ...
(Reading database ... 5%
(Reading database ... 10%
(Reading database ... 15%
(Reading database ... 20%
(Reading database ... 25%
(Reading database ... 30%
(Reading database ... 35%
(Reading database ... 40%
(Reading database ... 45%
(Reading database ... 50%
(Reading database ... 55%
(Reading database ... 60%
(Reading database ... 65%
(Reading database ... 70%
(Reading database ... 75%
(Reading database ... 80%
(Reading database ... 85%
(Reading database ... 90%
(Reading database ... 95%
(Reading database ... 100%
(Reading database ... 37439 files and directories currently installed.)
Preparing to unpack .../zsh-common_5.8-5_all.deb ...
Unpacking zsh-common (5.8-5) ...
Selecting previously unselected package zsh.
Preparing to unpack .../archives/zsh_5.8-5_arm64.deb ...
Unpacking zsh (5.8-5) ...
Selecting previously unselected package libevent-2.1-7:arm64.
Preparing to unpack .../libevent-2.1-7_2.1.12-stable-1_arm64.deb ...
Unpacking libevent-2.1-7:arm64 (2.1.12-stable-1) ...
Selecting previously unselected package tmux.
Preparing to unpack .../archives/tmux_3.1c-1_arm64.deb ...
Unpacking tmux (3.1c-1) ...
Selecting previously unselected package armbian-zsh.
Preparing to unpack .../armbian-zsh_21.02.0-trunk_all.deb ...
Unpacking armbian-zsh (21.02.0-trunk) ...
Setting up zsh-common (5.8-5) ...
Setting up libevent-2.1-7:arm64 (2.1.12-stable-1) ...
Setting up zsh (5.8-5) ...
Setting up tmux (3.1c-1) ...
Setting up armbian-zsh (21.02.0-trunk) ...
Processing triggers for man-db (2.9.3-2) ...
Processing triggers for libc-bin (2.31-9) ...
qemu: uncaught target signal 11 (Segmentation fault) - core dumped
Segmentation fault (core dumped)
qemu: uncaught target signal 11 (Segmentation fault) - core dumped
Segmentation fault (core dumped)
dpkg: error processing package libc-bin (--configure):
 installed libc-bin package post-installation script subprocess returned error exit status 139
Errors were encountered while processing:
 libc-bin
E: Sub-process /usr/bin/dpkg returned an error code (1)

[balbes150]

Disabled the installation of the zsh package and the images began to be collected to build on x86.

[Igor]

On 1/12/2021 at 9:11 AM, balbes150 said:

Perhaps then it is worth having a minimal configuration (set of packages) for DE ? That is, the minimum configuration describes (includes in the assembly) only the components that are necessary to form the  DE ?

Each desktop can have multiple configurations, but l would rather not make lots of them for start. Let's see where we need more than one. We need to keep our wishes under control  At least for a while until this whole new things is not stabilised. Currently bugs are still around and we find them every day.

 

On 1/12/2021 at 9:11 AM, balbes150 said:

officially supported DE

 

Should stay as similar as possible to current base, but later we can tweak it more. Someone mention an idea to salvage some visual tweaks from XUbuntu. To look nicer ...

 

On 1/12/2021 at 12:55 PM, balbes150 said:

Question. Is it possible to have different package settings for different architectures aarch64\arm64 (64 bits) and separately for armhf\arm (32 bits) ?

No. But you can have a desktop that is available only for arm64 and is disabled under arm32. What would be the purpose of this?

 

On 1/12/2021 at 9:11 AM, balbes150 said:

Perhaps to simplify control (so as not to check which mandatory groups are included when building the official version of DE) it is easier to have a fixed configuration that is selected automatically when building official versions, for example config_armbian (with a description of all packages and applications in one place\file)?

I think this whole part was not fully integrated. Currently its solved this way:
https://github.com/armbian/build/blob/desktop/config/targets-desktop-beta.conf

nanopct4            current         focal       desktop                  beta            yes           budgie  config_base   browsers chat desktop_tools editors email internet internet-tools multimedia office programming remote_desktop

 

On 1/15/2021 at 10:55 AM, balbes150 said:

Strange behavior of the CLI image build. When natively building an image for p1 (with any kernel, desktop branch), everything goes without errors and I get CLI images (sid bullseye focal), but when trying to build on x86, I get this error for sid bullseye focal, for hirsute the build goes without an error. I need to understand if this is my local problem (my build environment on x86) or if there is such a problem.

Didn't happened to me yet.

 

On 1/15/2021 at 1:05 PM, balbes150 said:

Disabled the installation of the zsh package and the images began to be collected to build on x86.

 

Strange.

[Steele Price]

anyone testing Bullseye yet?

I am using NanoPC-T4 and will be working on a system that builds Wayland/Weston and Sway so it also needs Debian Bullseye.
It will also be preinstalling dotnet as we are using that for Blazor and gRPC Projects.

[Igor]

11 hours ago, Steele Price said:

anyone testing Bullseye yet?

 

Currently broken 

 

[balbes150]

14 часов назад, Steele Price сказал:

anyone testing Bullseye yet?

Yes

[balbes150]

I tested the native build of the image using Bullseye (XFCE) on the latest DESKTOP branch . The minimal (basic) configuration works without errors.

For the full configuration (when all grupps are checked), you need to remove the libreoffice-style-tango package.

I removed the link to the BUSTER package, copied the file package, and deleted the libreoffice-style-tango package. The full build is error-free.

[Steele Price]

On 1/25/2021 at 6:00 AM, balbes150 said:

I tested the native build of the image using Bullseye (XFCE) on the latest DESKTOP branch . The minimal (basic) configuration works without errors.

For the full configuration (when all grupps are checked), you need to remove the libreoffice-style-tango package.

I removed the link to the BUSTER package, copied the file package, and deleted the libreoffice-style-tango package. The full build is error-free.

That is terrific news.  I just started working on a few things like getting everything encrypted with cryptsetup for lvm and luks, that's all working even with encrypted initramfs on a 970 Evo+.
We don't want Office anyway, looking for a minimal UI with just a Compositor so we can use PWAs and gRPC.

[Werner]

1 hour ago, Steele Price said:

everything encrypted with cryptsetup for lvm and luks,

Just curios. Did you create your build using the CRYPTROOT_* build flags or from scratch?

[Steele Price]

from scratch... I'm working out the exact process now to automate it.  When I have the scripting worked out, I'll format it into proper Armbian format and submit a PR. It was all pretty straight forward to accomplish manually.  Wipe the NVMe and setup several logical partitions to use as encryption stores, add them to fstab, etc. It's all working now.  My biggest gripe is which kernel is supported... so far this is only working on legacy 4.4 and I really want to use 5.7+

I have Wayland/sddm/Sway working as well and will be integrating that as my default managers.

My needs are a little different, I have a fairly tight device in the making.

[balbes150]

25.01.2021 в 16:00, balbes150 сказал:

I tested the native build of the image using Bullseye (XFCE) on the latest DESKTOP branch . The minimal (basic) configuration works without errors.

For the full configuration (when all grupps are checked), you need to remove the libreoffice-style-tango package.

I removed the link to the BUSTER package, copied the file package, and deleted the libreoffice-style-tango package. The full build is error-free.

Checking the latest GIT (DESKTOP branch), shows the same error. The SID and Bullseye network reps do not have this package  libreoffice-style-tango . If necessary, I can send a PR with a correction.

[balbes150]

question. Currently, the rockchip64-current kernel config has options for other platforms (AW AML iMX), this config is used exclusively for RK, why waste resources on unnecessary options ? There may have been plans to make a universal configuration, but then it makes sense to make a separate file that can be used for general configurations.

[Igor]

On 3/1/2021 at 1:16 PM, balbes150 said:

why waste resources on unnecessary options ?

Humans are biggest resource we have to spare. Compilation time is not an issue and I doubt there is noticeable difference. 

 

On 3/1/2021 at 1:16 PM, balbes150 said:

There may have been plans to make a universal configuration, but then it makes sense to make a separate file that can be used for general configurations.

We could link them together where possible and unlink if changes are needed. Also some more advanced logic could be enforced but that would require some RFC. IMO not a critical problem / can wait.

[balbes150]

13 часов назад, Igor сказал:

Humans are biggest resource we have to spare. Compilation time is not an issue and I doubt there is noticeable difference. 

Our kernel build system is now used by a lot of users not mediocre on the devices themselves. For them, when the native kernel is built on the device itself, the extra options in the configuration result in a few extra hours. Moreover, the optimization is very simple, copy the current configuration file with a new name, disable unnecessary platforms in the current rockchip64. If necessary, I can send a PR if there is a decision on which name to use for the general kernel configuration, or I can use something like "arm-64".

[Igor]

50 minutes ago, balbes150 said:

For them, when the native kernel is built on the device itself, the extra options in the configuration result in a few extra hours.

So they will come and help with maintaining?  If not, we will not complicate our lives more than we already do. On a cost of the time our "customers" lost. If their time is critical, they will support us properly.

Adding and splitting configs is simple task, agree, like most of small development tasks. But from the maintaining point of few, this is another set of configurations which needs attention and needs to be maintained. Since must of such dirty work nobody sees falls on me and since there is virtual no interest to help out https://forum.armbian.com/forum/54-help-wanted/ I can't support ideas that bring more on maintenance work.

[balbes150]

1 час назад, Igor сказал:

Adding and splitting configs is simple task, agree, like most of small development tasks. But from the maintaining point of few, this is another set of configurations which needs attention and needs to be maintained. Since must of such dirty work nobody sees falls on me and since there is virtual no interest to help out https://forum.armbian.com/forum/54-help-wanted/ I can't support ideas that bring more on maintenance work.

Here is just the opposite situation. By removing unnecessary (unused) options, you get simplified support and reduced costs. Extra options cause more compatibility problems when a broken driver of one platform breaks the work of others and you spend time figuring out why the main device does not work. Fewer options = fewer build issues when changes to the source code are not compatible with the current patches. This is the law of programming, the fewer elements (code in processing), the fewer errors in the output and easier support (no cross-influences).

I don't insist, it's just information.

[Igor]

1 minute ago, balbes150 said:

Fewer options

I agree with a theory. But if we remove any random driver, someone (or more of them) will get on a forum and start complaining about. It is in human nature they will ask us for explanation why this was removed. 

 

We removed certain thing in the past by accident and almost always resulted in yet another bulk of time we lost + stress due to urgency. I don't mind if that time would be paid so we can easily level the stress down by hiring someone to do extra things. Since that is not an option ... 

[kentAVR]

After good news about Panfrost support the main question is
Does armbian support wayland since hirsute?

 

I thinked

-Of cause yes. Ubuntu hirsute is switched to wayland, but

 

I look at /build/config/desktop/hirsute/environments/gnome/config_base/packages

xserver-xorg xserver-xorg-video-fbdev gvfs-backends xfonts-base xinit bubblewrap dbus-x11 dictionaries-common evolution-data-server ...

 

I dislike gnome. Let's go to /build/config/desktop/hirsute/environments/xfce/config_base/packages

xserver-xorg xserver-xorg-video-fbdev xfonts-base xinit x11-xserver-utils xinit xorg-docs-core x11-apps xscreensaver xfonts-100dpi

 

So Wayland or X.Org? Looks like X.Org...