Question | H3 unified imaging?

[Fourdee]

Hi all,

 

I'am looking at creating a single unified H3 DietPi image (currently using ARMbian build tools), rather than one for each board. Would allow us to support all H3 devices, reduce work load and simplify DietPi source code.

Basically I need to know if its possible, and which unique files/configurations would need changing to achieve it.

 

The end goal:

The user would download the H3 image, modify it for their board (eg: copy and paste job on /boot), prior to powering on.

 

Some questions:

• Is script.bin the only unique configuration across all H3 boards? Aside from any additional linux stuff, like hardware items (eg: + onboard BT/WiFi)
• Does /usr/lib/linux-u-boot-orangepione_5.17_armhf/*.bin have any effect on the configuration of the board during boot, doesn't script.bin do this already?
• Could this be done across all sun8i devices?
• Is there a way we can identify which board we have from within linux? eg: OPi PC. Thinking a service could be created during 1st run to copy the required script.bin to /boot/script.bin automatically. Would require a "compatible" script.bin that could safely be used on all H3 devices.

Thanks in advanced.

 

Kind Regards,

Dan

[jernej]

There is also dtb file for U-Boot, which is integrated inside it. You could probably make a common subset for that, but then you would need to find other ways to do some things like power on USB to SATA bridge and other stuff.

[tkaiser]

Sounds like you're missing a few of the more important details here. What happens in u-boot is very important since some stuff won't be touched by the kernel at all. Of course board configuration differs and here and there and that's also important (appropriate settings). And if it would be possible to use one image for all H3 devices, then... why shouldn't Armbian do it exactly that way (we've been there, it didn't work and it took us months to revert back from this approach).

 

Finally: when I read DietPi's project goal I wonder why you're not already running kernel 4.7 and rely on schedutil cpufreq governor (again: ondemand is wrong for Allwinner BSP kernels)

[Fourdee]

why shouldn't Armbian do it exactly that way (we've been there, it didn't work and it took us months to revert back from this approach).

 

Well, that answers that question then. Thanks for info

 

 

 

Finally: when I read DietPi's project goal I wonder why you're not already running kernel 4.7 and rely on schedutil cpufreq governor (again:ondemand is wrong for Allwinner BSP kernels)

 

http://dietpi.com/

Lightweight justice for your single-board computer.

Optimized | Simplified | For everyone

 

Kernel 4.x+:

• Tried a few months back with various boards using ARMbian build tools. Devices wouldn't boot. Haven't tried since.
• If I remember correctly. Your website used to list issues when using the "vanilla kernel", reduced support and features? Put me off trying further.

 

ondemand is wrong for Allwinner BSP kernels:

One could say, Allwinner H3 is wrong  . On your experience, is interactive "right" for Allwinner?

[ssvb]

ondemand is wrong for Allwinner BSP kernels:

One could say, Allwinner H3 is wrong  . On your experience, is interactive "right" for Allwinner?

You can find some information about cpufreq governors here: https://linux-sunxi.org/Cpufreq

 

Basically, the "ondemand" governor is way too slow to increase CPU clock speed. If you want to run a graphics desktop, then it feels rather laggy and unresponsive. Obviously, it was not fit for Android, and that's why Android people have implemented the "interactive" governor, which is admittedly also far from perfect, but at least it is nowhere as bad as "ondemand". The shortcomings of the "ondemand" governor have been known since a long time ago, but only now the "schedutil" governor is trying to implement it in a proper way.

[ssvb]

Hi all,

 

I'am looking at creating a single unified H3 DietPi image (currently using ARMbian build tools), rather than one for each board. Would allow us to support all H3 devices, reduce work load and simplify DietPi source code.

Basically I need to know if its possible, and which unique files/configurations would need changing to achieve it.

It's not quite related to ARMbian build tools, but it is technically possible to have a generic bootable image / installer for all Allwinner devices. I had implemented a prototype a long time ago:http://linux-sunxi.org/SD_card_based_installer

 

The sunxi-bootsetup demo only supports a set of A10/A10s/A20 boards, but in principle all the Allwinner chip generations can be covered (including H3). For example, you can try a simple helloworld binary, which can be written to the SD card instead of a bootloader (the source code is also available in github):

 

wget https://github.com/linux-sunxi/sunxi-tools/raw/master/bin/uart0-helloworld-sdboot.sunxi
dd if=uart0-helloworld-sdboot.sunxi of=/dev/sdX bs=1024 seek=8

 

Unfortunately none of the Linux distribution maintainers was interested in this feature back in 2014, probably it was way too unorthodox for them to accept. And of course, later the Raspberry Pi people have introduced an universal SD card image of their own, which can boot on all Raspberry Pi generations. Just because they probably do care about their users a little bit more

[nz1]

How often would it occur that the usb keyboard wouldn't be recognised?

What about adding this to a pc and then connecting the sbc to the pc with a usb cable and then using fel boot fallover.

Then the usb keyboard problem would go away and the user could select the target board from a list?

My 2cents. Your implementation/prototype looks impressive.

[tkaiser]

It's not quite related to ARMbian build tools, but it is technically possible to have a generic bootable image / installer for all Allwinner devices. I had implemented a prototype a long time ago:http://linux-sunxi.org/SD_card_based_installer

 

The 'problem' with this from an Armbian point of view is that we don't want to rely on a setup where the user needs keyboard + display to do the initial setup. It should work in a way where the device boots up on its own and is then accessible through SSH afterwards. So auto detection would be ok but no user interaction to let the user choose his device (BTW: it happened not just once that users try the wrong OS images for their boards and discover after lengthy threads that they do not own an Orange Pi PC but an OPi 2 instead).

 

With some devices auto detection could work reliable but with most of them not (eg. testing for the existence of an internal Terminus USB hub and GbE PHY to detect whether it's an OPi Plus or Plus 2 fails since users do not follow the recommendation to unplug every USB peripherals on first boot and then an OPi Plus 2E will be wrongly identified).

 

And the horror continues: if we want to support NanoPi AIR we have to tweak the OS image for this device in a way that it's accessible through USB serial gadget like we do it already with Orange Pi Lite (no UART pins soldered, no HDMI, no USB ports other than OTG which is used to power the device, no Ethernet).

 

For the AIR I also thought about enhancing our toolchain, providing utilities for Windows, Linux and OS X relying on https://github.com/zador-blood-stained/fel-mass-storage and then simply allowing the user to pre-fill WiFi authentication data. But that would require Armbian images that use a small FAT partition since mounting ext4 is IMO too unreliable on Windows and OS X.

[Fourdee]

 

 

And the horror continues: if we want to support NanoPi AIR we have to tweak the OS image for this device in a way that it's accessible through USB serial gadget like we do it already with Orange Pi Lite (no UART pins soldered, no HDMI, no USB ports other than OTG which is used to power the device, no Ethernet).

 

pre-fill WiFi authentication data. But that would require Armbian images that use a small FAT partition since mounting ext4 is IMO too unreliable on Windows and OS X.

 

We have a system in DietPi that allows users to pre-enter their wifi details in /boot/dietpi.txt before powering on. By default, all DietPi images have all networking disabled, then this code runs after resize: 

https://github.com/Fourdee/DietPi/blob/master/dietpi/boot#L115-L213 

 

Config file, located on FAT partition:

https://github.com/Fourdee/DietPi/blob/master/dietpi.txt#L4-L24

 

Default /etc/network/interfaces:

https://github.com/Fourdee/DietPi/blob/master/dietpi/conf/network_interfaces

 

So, ARMbian could have a similar system, config /boot/armbian.txt, loaded during 1st run. I'll fork ARMbian and add this system in.

 

 

 

 

Unfortunately none of the Linux distribution maintainers was interested in this feature back in 2014, probably it was way too unorthodox for them to accept. And of course, later the Raspberry Pi people have introduced an universal SD card image of their own, which can boot on all Raspberry Pi generations. Just because they probably do care about their users a little bit more

Yep, have to hand it to RPi, although ARMv6 package and Repo, compatibility and simplicity across all their devices is exceptional.

I'll take a look at your notes for your prototype installer, thanks for sharing. Although, if i'am honest, might be out of my depth at the moment. When it comes to uboot/initrd, I use ARMbian to sort it out lol

[tkaiser]

So, ARMbian could have a similar system, config /boot/armbian.txt, loaded during 1st run.

 

Of course, this is more or less what I wrote above but that would require the use of a separate FAT partition which is something we currently do not implement on most OS images since there was no necessity for it (unlike RPis where it's mandatory since VideoCore 4 can not deal with anything else than FAT).

 

I just put together an USB flashing tool for H3 devices connected to OS X hosts (based on Zador's fel-mass-storage preparations) just to realize that NanoPi AIR has also no FEL button

 

So in case FriendlyARM ships these devices with a pre-populated eMMC there's no easy way without flashing a separate SD card.

[zador.blood.stained]

I just put together an USB flashing tool for H3 devices connected to OS X hosts (based on Zador's fel-mass-storage preparations) just to realize that NanoPi AIR has also no FEL button

 

So in case FriendlyARM ships these devices with a pre-populated eMMC there's no easy way without flashing a separate SD card.

There is fel-sdboot.sunxi image here, so you can use universal SD card that can boot any supported board in FEL mode. I guess it's better than nothing.

[tkaiser]

There is fel-sdboot.sunxi image here, so you can use universal SD card that can boot any supported board in FEL mode. I guess it's better than nothing.

 

Sure but it's a bit sad that our users have to use ugly workarounds like these (and it seems there are not even pins available to enter FEL mode like on C.H.I.P. for example). BTW: The OS X application wrapper I built is simply just sunxi-fel with libusb-1.0.0.dylib and fel-mass-storage lying next to it. And the whole logic is just that:

 

 

#!/bin/bash
#
# DESCRIPTION
#
# This is an application wrapper for sunxi-fel tool and accompanied u-boot + kernel
# to fel boot Allwinner devices (H3 the only SoC being supported at the moment).
#
# CHANGES
#
# 1.0: - first version.
#
# TODO
#
# - add other Allwinner variants

Main() {
	export PATH=/bin:/usr/bin:/usr/local/bin
	# MyTmpFile="$(mktemp /tmp/${0##*/}.XXXXXX || (echo -e "Not able to create temporary file below /tmp/. Exiting." | logger ; exit 1))"
	# trap "rm -f \"${MyTmpFile}\"; exit 0" 0 1 2 3 15
	
	PathToFEL="${0%/*}"/sunxi-fel
	PathToKeys="${0%/*}"/keys
	
	sleep 0.2
	IsShiftPressed=$("${PathToKeys}" shift)
	IsOptionPressed=$("${PathToKeys}" option)

	# case ${IsShiftPressed} in
	# do some fancy stuff based on modifier key pressed or not
	
	CountOfSunxiDevices=$("${PathToFEL}" ver 2>/dev/null | wc -l)
	echo -e "Searching for Allwinner devices appearing on USB bus...\c"
	while [ ${CountOfSunxiDevices} -ne 1 ]; do
		sleep 0.5
		CountOfSunxiDevices=$("${PathToFEL}" ver 2>/dev/null | wc -l)
	done
	
	echo -e "\n"
	StartFlashing
	
	# start Etcher if installed
	open -b io.resin.etcher 2>/dev/null || echo -e "\nYour Allwinner device should now appear as an USB stick."
} # Main

StartFlashing() {
	FELIdentifier="$("${PathToFEL}" ver)"
	case ${FELIdentifier} in
		*soc=00001680*)
			echo -e "Found H3 device: ${FELIdentifier}"
			BOARD=h3
			"${PathToFEL}" -p uboot "${0%/*}/fel-mass-storage/$BOARD"/u-boot-sunxi-with-spl.bin \
				write 0x42000000 "${0%/*}/fel-mass-storage/$BOARD"/zImage \
				write 0x43000000 "${0%/*}/fel-mass-storage/$BOARD"/script.bin \
				write 0x43300000 "${0%/*}/fel-mass-storage/$BOARD"/uInitrd \
				write 0x43100000 "${0%/*}/fel-mass-storage/$BOARD"/boot.scr
			;;
	esac
} # StartFlashing

Main "$@" 

 

 

 

Simple GUI app that could've been extended to query network configuration (asking configd for available WiFi networks and even getting the WiFi passphrase from OS X' keychain and so on). But ATM it's rather limited if we have trouble getting some eMMC equipped devices into FEL mode.

[ssvb]

The 'problem' with this from an Armbian point of view is that we don't want to rely on a setup where the user needs keyboard + display to do the initial setup. It should work in a way where the device boots up on its own and is then accessible through SSH afterwards. So auto detection would be ok but no user interaction to let the user choose his device (BTW: it happened not just once that users try the wrong OS images for their boards and discover after lengthy threads that they do not own an Orange Pi PC but an OPi 2 instead).

The user interaction part is optional. We can easily tweak the code not to require any confirmation when there is only a single suitable choice available. For example, if the installer is only targeting Cubieboard1 / Cubieboard2 / Cubietruck, then there is just a single choice for the user (approve the installation or don't do it).

 

As for the users doing a wrong choice, this is a universal problem and no better solution exists anyway. If you have multiple SD card images on the download page instead of a unified one, then the user still can download a wrong image and use it. The unified installer is in fact reducing the chance of a user error, because it does not allow the user to select Cubieboard2 when his/her SoC is detected as Allwinner A10 at runtime Moreover, the unified installer tries to provide instructions to the user at each step and explains how to make a correct choice.

 

But this all is an ancient history Fast forward to the present days. Now I think that a proper solution is to have the exact board identification information available in the SPI flash or eMMC. There was some discussion about this in the Olimex blog: https://olimex.wordpress.com/2016/09/01/a64-olinuxino-emmc-rev-b-oshw-64-bit-arm-development-board-prototypes-are-testing/

 

And I suggested a patch for U-Boot to make the device model identification easy: https://patchwork.ozlabs.org/patch/629172/

 

The 'problem' with this from an Armbian point of view is that we don't want to rely on a setup where the user needs keyboard + display to do the initial setup. It should work in a way where the device boots up on its own and is then accessible through SSH afterwards. So auto detection would be ok but no user interaction to let the user choose his device (BTW: it happened not just once that users try the wrong OS images for their boards and discover after lengthy threads that they do not own an Orange Pi PC but an OPi 2 instead).

It depends on the intended use of the device. If the user wants to run a desktop Linux system on the device, then we can expect a USB keyboard and a HDMI monitor to be normally available. You seem to be talking about the headless use case here. And it may (or even should?) have a different installer, or just the plain and boring board-specific SD card images.

 

And the horror continues: if we want to support NanoPi AIR we have to tweak the OS image for this device in a way that it's accessible through USB serial gadget like we do it already with Orange Pi Lite (no UART pins soldered, no HDMI, no USB ports other than OTG which is used to power the device, no Ethernet).

 

For the AIR I also thought about enhancing our toolchain, providing utilities for Windows, Linux and OS X relying on https://github.com/zador-blood-stained/fel-mass-storage and then simply allowing the user to pre-fill WiFi authentication data. But that would require Armbian images that use a small FAT partition since mounting ext4 is IMO too unreliable on Windows and OS X.

This seems to be a very specialized device. The users of such device are expected to know what they are doing.

[ssvb]

Of course, this is more or less what I wrote above but that would require the use of a separate FAT partition which is something we currently do not implement on most OS images since there was no necessity for it (unlike RPis where it's mandatory since VideoCore 4 can not deal with anything else than FAT).

 

I just put together an USB flashing tool for H3 devices connected to OS X hosts (based on Zador's fel-mass-storage preparations) just to realize that NanoPi AIR has also no FEL button

 

So in case FriendlyARM ships these devices with a pre-populated eMMC there's no easy way without flashing a separate SD card.

Well, it was Allwinner who introduced their reference design, which relies on having a FEL button and doing firmware upgrade/recovery over the Micro-USB cable using their LiveSuit software. Many tablets and other devices are compatible with this reference design. And it looks like you and Zador are implementing your own open source alternative to LiveSuit.

 

And of course, a few ugly ducklings or very specialized devices exist. They may need some special treatment.