espressobin completely set aside?

[ABF023]

😭

[Igor]

This device does not meet basic support criteria anymore:
https://docs.armbian.com/User-Guide_Board-Support-Rules/

 

Start here: https://docs.armbian.com/Process_Contribute/ and start maintaining this device or find someone (outside Armbian team) that will be doing this instead of you. When you fix problems, share it with others https://github.com/armbian/build/pulls ... sent fixes to upstream u-boot / kernel, or not. Our resources are way too small and your support is virtually non-existing. We can't support all hardware that exists. And especially not indefinitely and with complete lack of funding. Enjoy open source and stop blaming us who invested into your wrong purchase. Make a donation instead. For all the years of software you didn't have troubles with: https://www.armbian.com/donate/ My reply sounds ridiculous, I know, but this is how it is.

[ABF023]

I understand your decision, thank you for your continuous support🌻🌻

[Igor]

5 hours ago, ABF023 said:

I understand your decision, thank you for your continuous support

Deprecating some hardware is always hard, depressing and at the edge of desperation 😢 Lets hope that someone picks it up from here ...

[bschnei]

@ABF023 while I don't own this device, I do own a variant (ESPRESSObin Ultra). I can't speak for the previous device maintainer (@ManoftheSea), but having spent a lot of time working on the bootloader for my device, I can tell you one of the biggest obstacles this family of devices (Armada 37xx/mvebu) faces with getting any sort of Linux distribution support is that they are shipped from Globalscale with a bootloader that doesn't support modern (UEFI) booting.

 

This is all the more frustrating because upstream U-Boot (the bootloader used by this family of devices) DOES support UEFI. Globalscale's bootloader repositories are open source, but haven't been updated in years. So, as you'll see, I forked their build repo and did a lot of work to bring everything up-to-date which fixes the issue of UEFI not being supported by the factory bootloader.

 

Indeed, you'll note on the device page that "manual flashing to latest u-boot is mandatory" along with a scary note about instability.

 

I announced my work in this forum a while back, but nobody has reached out. While my repo builds firmware specifically for the Ultra variant of the device, it shouldn't be too complicated to modify the scripts and configuration to build for other A37xx variants (assuming upstream U-Boot supports the device).

 

I personally do not have capacity to provide "soup to nuts" support for this device on Armbian, but if anyone were interested in picking up where the previous maintainer left off and wanted help building a modern bootloader that would make supporting this family of devices much easier for any maintainer, then I would be happy to help.

 

EDIT: FYI, I believe this family of devices also gets some support from both Debian and Arch Linux ARM communities. Note that Arch Linux ARM is currently independent of the Arch Linux project, though there are some initial discussions happening around how Arch Linux could provide better support for non-x86 architectures.

Edited July 17 by bschnei

[Igor]

16 hours ago, bschnei said:

while I don't own this device, I do own a variant (ESPRESSObin Ultra)

 

Ebin Ultra is very similar as this one, I even have one device somewhere at home, introducing several new troubles ... I think it even boots and works to some degree with normal Ebin image without changes.

 

16 hours ago, bschnei said:

bootloader that doesn't support modern (UEFI) booting.

 

This is pretty normal in this world. Most of custom hardware does not have UEFI nor need it. UEFI is expensive to make and they sell hardware at BOM + profit only. Software support is buyers problem at the end.

 

16 hours ago, bschnei said:

it shouldn't be too complicated to modify the scripts and configuration to build for other A37xx variants (assuming upstream U-Boot supports the device)

Time is the problem. I can spent sharing some experiences, not sure if valuable ... We have lost insane amount of time  in past years trying to keep functional Linux image for this hardware. User space plays absolutely no role, kernel little.

 

And this is just one custom hardware out of many. We received hardware valued 50 USD as a support for this in all those years in exchange for hundreds of hours to keep this device usable ... and Armbian team mainly don't need this to work. Users do, while costs of keeping this (terrible made and supported) is over extreme for our hobby / private time / pockets. And users, which are in 99% unable to see pain and frustration, mainly complains how something doesn't work ... never happy people never rewards maintainers. 

 

16 hours ago, bschnei said:

I would be happy to help

 

Work here is "only" related to keeping boot loader functional. Which is ofc not small nor simple. From there on,

 

16 hours ago, bschnei said:

I believe this family of devices also gets some support from both Debian and Arch Linux ARM communities.

 

standard generic aarch64 kernel boots. For several years now. I don't think we can count anything from end users / amateur distributions as they require standard way of booting, but there was help from kernel.org communities.

As this board can be moved under generic kernel, so whole marvel SoC mvebu64 kernel family can be dropped. Which would need to be expense on overloaded Armbian team. Or good will of someone from community.

 

My 2c.

 

[bschnei]

No disagreement @Igor. I wouldn't recommend any distro attempt to maintain device-specific support for this family of devices. Especially because there is a pathway to UEFI booting which does allow these devices to be supported by generic kernel packages. I also feel that it shouldn't land on any Linux distribution to be expected to provide support for bootloaders. As a result, I was impressed by the amount of effort I saw around this device and sympathize with how you and the Armbian team must feel.

 

I would, however, caution against letting that frustration spill over into resentment of your user base (or potential user base). I'm not familiar with Armbian's philosophy and what the expectations are between users and maintainers/developers. At Arch Linux the philosophy and expectation is extremely DIY/self-support oriented and those expectations are made very clear. However, that can come across as rude/off-putting to people who show up looking for help. And when you have a volunteer effort (which by definition means you are chronically under-resourced), that kind of philosophy/approach can scare away potential future volunteers/contributors. And obviously if a volunteer project can't attract/retain new contributors then it's just a matter of time before it's dead. Anyone considering Arch Linux ARM for this family of devices would be wise to read this thread: https://bbs.archlinux.org/viewtopic.php?id=290931

 

Personally, I throw most of the blame for the state of these devices on Globalscale and Marvell. The CPU frequency scaling debacle that seems to actually be related to an improperly configured bootloader all along seems to have burnt everyone out so people moved on to other devices. The one silver lining is the open source bootloader. With frequency scaling no longer an issue and UEFI supported, there really is no technical reason these devices can't be supported:

 

ben@ebu-dev:~$ sudo hostnamectl
 Static hostname: ebu-dev
       Icon name: computer-embedded
         Chassis: embedded
Operating System: Arch Linux ARM                             
          Kernel: Linux 6.9.9-1-a3700
    Architecture: arm64
 Hardware Vendor: globalscale
  Hardware Model: Globalscale Marvell ESPRESSOBin Ultra Board
Firmware Version: 2024.04-dirty
   Firmware Date: Mon 2024-04-01
    Firmware Age: 3month 2w 3d

 

It is, as you correctly note, a question of time/effort.

 

I'm not super familiar with this distribution and community and don't want to step on anyone's toes. If the generic kernel packages provided here are still being configured to support these devices and the only obstacle is how to fix/configure the bootloader, then that's where I am offering to share my experience. But anyone that wants to get Armbian running on their mvebu device is going to have to be willing to put in the work to at least get their hands dirty messing with U-Boot and potentially even building/flashing a more modern bootloader.

[Igor]

19 minutes ago, bschnei said:

I'm not familiar with Armbian's philosophy

Armbian aim its to make and maintain Linux (firmware) from when the power is connected to the board. This is much closer to embedded Linux then normal Linux distributions. Nothing less then full control. Ofc this is not always possible, but we know how to do it ... and when / where this is achieved, this is fun. That heals frustrations and depression   Problem starts with maintaining all this. As this is everything but fun, require different mindset, gets boring and is easy developed into full time job that brings no / too little food on the table. Average (x86/Rpi) Linux distro users, who doesn't understand any of this, would "only like that everything works".

 

Armbian is Debian from outside, but it could / can be Arch or whatever flavor, if someone would be crazy enough / have have too much time to add extension to make Arch output / rework that that kind of customization are possible. Why doing that if the aim is to run Linux firmware / some use case / and that is already very difficult as this everything but standard made hardware. Vendors don't want that you run (mainline) Linux. They want to sell hardware with their partially closed SW, similar to / with Android.

We want to help users running Linux on those exotics, often for single purpose made, devices. In embedded / custom Linux world, where we are, distribution philosophies are usually wrong. Yes, you / we would like to have UEFI to spare troubles ... well, making it is Concorde fallacy by default except mainstream hw. Do you have required funds and / or enough power to motivate volunteers to make it? Users are scattered around many many devices, which will (never) be under UEFI. If you move device to mainline Linux with basic functions, its already a (great) success. Costs millions (core work can not be done and its not done by volonteers) on top and around community help. Where masses are behind - Rpi, now RK3588 and similar variants are going to get this, others too late or never ...

 

22 minutes ago, bschnei said:

I also feel that it shouldn't land on any Linux distribution to be expected to provide support for bootloaders.

Many things should exists but they do not  Making hardware was never this cheap and Rpi model proved that community will make firmware instead of them or not at all. Something is always provided by SoC maker. This is what they hope for. Some helps and invest into SW, some not at all. And most users have don't know this nor care about.

 

Armbian is Linux build framework first. A framework that helps us and community maintain this endless crowd of custom hardware. They were purposefully made not to be standard and what is under Armbian is already a lot of standardization applied. Because power of build framework and people that made and maintain this. Regulars, part timers, random people that shows up. We could do more, but we don't have re$ources or sufficient level of support.

 

1 hour ago, bschnei said:

then that's where I am offering to share my experience.

There are some people that will be very helpful for this, but usually they do not have the know-how to pick it from here. Volunteer developers do that anyway, but majority of solutions are never integrated as core team is simply too small. And this is never to be changed. We are surviving with keeping the focus on some devices, leaving others "to community / anyone that wants / upstream / * ". Line has to be set somewhere as otherwise doers would burn out, level of sadness would eat us out. Some exotic devices nobody will ever pick it up if some of developers won't donate months of his time. Then this works is picked up by some other distributions. We have noticed ArchLinux re-packing and serving our Rockchip kernel, others are too exotic.

 

1 hour ago, bschnei said:

I throw most of the blame for the state of these devices on Globalscale and Marvell.

This market function this way. They design hardware and provide support for business clients. Amateurs are often collateral damage.

 

1 hour ago, bschnei said:

expectations are made very clear. However, that can come across as rude/off-putting to people who show up looking for help.

 

This is hard to change. Expectations are set with marketing and are often set with different objectives. Then you add complete lack of technical understanding from end users and you get a clash. People that come for help are frustrated, support givers are 1/1000 and totally milked out ...

 

1 hour ago, bschnei said:

And when you have a volunteer effort (which by definition means you are chronically under-resourced), that kind of philosophy/approach can scare away potential future volunteers/contributors.

 

Armbian supports (in term of ordinary Linux distro) lets say 200 devices. In reality we only know for those, where we have maintainers - "standard Armbian support"

https://www.armbian.com/download/?device_support=Standard support which are volunteer and they come and go. We (core team, partially pro) maintain framework, ecosystem, some devices. They don't maintain devices for us (Armbian), they maintain devices for you, users, community. Device that works good with Armbian, can easily run any other distro. Well, not UEFI easy. Still some devices were made in small quantities or were largely abandoned for better new models. Many people run do-not-touch-as-it-works glued together image which nobody can ever recompile. This problem Armbian solves very well. And closer to humans as some industrial grade tool such as Yocto.

[bschnei]

Thanks for the taking the time to provide this additional context! If full device support (i.e. including bootloader) is the goal, that is indeed an ambitious one.

 

I'll do what I can to offer bootloader related support where I can in the forums here, but my advice to everyone is generally going to be to move to UEFI/U-Boot Standard Boot (`bootflow scan`). Some of the U-Boot scripting I've seen to try and support all of the possible combinations of hardware and potential removable storage is just too clunky. When users show up with devices that won't boot, it's a true nightmare to try to support. As a result, any configuration where U-Boot loads additional configuration from a  storage device (vs its dedicated SPINOR) is always going to problematic because they vary across devices in the same mvebu family and users may want to change them (e.g. boot partition on USB device vs EMMC or SATA).

 

In short, U-Boot should ideally only chainload a userspace configurable bootloader (systemd-boot, GRUB, etc.) that can be more easily updated/packaged and supported.

[Igor]

14 hours ago, bschnei said:

Thanks for the taking the time to provide this additional context! If full device support (i.e. including bootloader) is the goal, that is indeed an ambitious one.

I am glad you are able to understand. 

 

14 hours ago, bschnei said:

Some of the U-Boot scripting I've seen to try and support all of the possible combinations of hardware and potential removable storage is just too clunky

 

Yeah, its done different for each device, untested on all combo and stressful to support. In embedded Linux you need to support specific use case and you tune u-boot to support that ...

 

14 hours ago, bschnei said:

In short, U-Boot should ideally only chainload a userspace configurable bootloader (systemd-boot, GRUB, etc.) that can be more easily updated/packaged and supported.

 

Oh, we are well aware  Here is, perhaps little outdated, attempt to come up with something better: