Armbian boot process unification

chradev · July 14, 2016

Hi to All,

I decided to start this new tread for searching of the best solution for unification of the boot process from any available media.

Main target is Armbian next and development branches and mainline Kernel and U-Boot.

Boot device can be eMMC, SD/MMC, USB Flash/HDD and SATA HDD/SSD probed in given order.

In case of booting from eMMC, SD/MMC and USB Flash Read-Only Root FS can be used optionally.

In case of booting from SD/MMC or USB Flash eMMC and / or SATA devices can be updated automatically.

Some effort in this direction was done by me and described in Armbian Customization thread.

The tests was done on Olimex' A20-Olimexino-Lime2-eMMC (HW rev. E) board with following additional staff:

60GB SATA SSD with SATA-CABLE-SET
MOD-WIFI-R5370 mounted on the rear panel - only antenna outside
USB-ETHERNET-AX88772B - optional
BATTERY-LIPO6600mAh
ALUMINIUM-HEATSINK-20x20x6MM later changed with special high efficient cooling system
A10-OLinuXino-LIME-BOX later changed to water and dust resistant solution based on Aluminum box
FTDI FT4232H based USB Hi-Speed Serial/Hub Module
AC/DC 5V/4A wall adapter
Dual LED Red + Blue Display Digital Voltmeter Ammeter DC 3.5-28V 3A
Additional User Push Button and 3 LEDs are connected via Lime2 GPIO-1 connector
1-Wire GPIO based Master and additional Device Boot control is implemented via Lime2 GPIO-1 connector
Reset, Power On/Off buttons and Charge LED are wired to front panel doubles
Console, DC-IN, USB and Ethernet connectors are wired to rear panel doubles

The current status is:

U-Boot is patched to:
- recognize eMMC;
- search a boot-able device on SATA, USB, SD/MMC, eMMC etc.;
- boot from the boot-able device with the highest priority;
- send to the kernel via the command line right root=PARTUUID=${uuid};
Kernel is patched to recognize eMMC;
firstrun script is modified to resize right MMC device (mmcblk0 or mmcblk1);
eMMC and SATA device experimental update scripts are implemented and tested;
it is investigated the classical and overlayfs way to run Debian on RO Root FS;
Described staff is working fine on latest Armbian (5.17), Kernel (4.6.3) and U-Boot (2016.05).

Not solved issues are:

to mount root accordingly without explicit setting in /etc/fstab;
to resize root partition from firstrun script in case of USB and / or SATA devices;
to prepare and use Read-Only Root FS on eMMC, SD/MMC and USB Flash optionally;
to start eMMC and /or SATA device update procedure from firstrun script if boot from SD/MMC or USB Flash.

The main use case will include following steps:

If the system will be setup for the first time SD/MMC has to be used for been able to install U-Boot on empty eMMC;
In case of next FW update USB Flash will be used to boot from by installed on eMMC U-Boot;
Booting from SD/MMC will be forced by pushing of user button or renaming of /boot/boot.scr file on the devices with higher priority;
Booting from USB Flash will be done always when present because of highest priority;
firstrun script will be executed always if booting from image cloned device and the system will reboot if necessary;
At the end of firstrun script eMMC and / or SATA device update scripts will be executed to update them from image or SD/MMC or USB Flash itself.

The following staff was used to customize Armbian:

Mainline U-Boot patch to enable eMMC:

Reveal hidden contents

Mainline U-Boot patch to change boot order:

Reveal hidden contents

Mainline U-Boot boot script:

Reveal hidden contents

Mainline Kernel patch to enable eMMC:

Reveal hidden contents

Change in do_expand_rootfs() function of firstrun script to recognize right MMC device to resize:

#	DEVICE="/dev/"$(lsblk -idn -o NAME | grep -w mmcblk0)
	DEVICE=$(mount|grep ' / '|cut -d' ' -f 1 | cut -dp -f1)

Reboot script and service looking for user button push:

Reveal hidden contents

#! /bin/bash
### BEGIN INIT INFO
# Provides: rebootd
# Required-Start: $sysfsutils
# Required-Stop: $sysfsutils
# Default-Start: 2 3 4 5
# Default-Stop:
# Short-Description: Banana Led disabler
### END INIT INFO

# Immediately exit if not called by init system
if [ "X$1" != "Xstart" ]; then
exit 1
fi

DAEMON=/tmp/rebootd

cat > $DAEMON <<'EOF'
#!/bin/bash
#
# blink red led if firstrun is still running
# listen to the reboot button push
# reboot when button is pushed
#

trap process_USR1 SIGUSR1
process_USR1() {
# echo 'Got signal USR1'
exit 0
}

me_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
me_FILE=$(basename $0)
cd /

if [ "$1" = "child" ] ; then # 2. We are the child. We need to fork again.
shift
umask 0
exec setsid $me_DIR/$me_FILE XXrefork_daemonXX "$@" /dev/null 2>/dev/null &
exit 0
fi
if [ "$1" != "XXrefork_daemonXX" ] ; then # 1. This is where the original call starts.
exec $me_DIR/$me_FILE child "$@" &
exit 0
fi

shift

sleep 30

if [ ! -f /sys/class/gpio/gpio87 ]; then
echo 87 > /sys/class/gpio/export
echo "out" > /sys/class/gpio/gpio87/direction
fi
if [ ! -f /sys/class/gpio/gpio88 ]; then
echo 88 > /sys/class/gpio/export
echo "out" > /sys/class/gpio/gpio88/direction
fi
if [ ! -f /sys/class/gpio/gpio50 ]; then
echo 50 > /sys/class/gpio/export
echo "in" > /sys/class/gpio/gpio50/direction
fi

RED_LED=/sys/class/gpio/gpio87/value
GREEN_LED=/sys/class/gpio/gpio88/value
RESET_BUTTON=/sys/class/gpio/gpio50/value

if [ -f /tmp/create_swap.sh ]; then
echo 1 > $RED_LED;
echo 0 > $GREEN_LED;
BLINKING_LED=$RED_LED;
else
echo 0 > $RED_LED;
echo 1 > $GREEN_LED;
BLINKING_LED=$GREEN_LED;
fi

while [ -f /tmp/create_swap.sh ]; do
sleep 0.5;
if [[ "$(cat ${BLINKING_LED})" == "1" ]]; then
echo 0 > $BLINKING_LED;
else
echo 1 > $BLINKING_LED;
fi
done
echo 0 > $RED_LED;
echo 1 > $GREEN_LED;
while [[ "$(cat ${RESET_BUTTON})" == "1" ]]; do sleep 1; done
echo 1 > $RED_LED;

echo 87 > /sys/class/gpio/unexport
echo 88 > /sys/class/gpio/unexport
echo 50 > /sys/class/gpio/unexport

EOF

echo -e "rm -f $DAEMON\nreboot\nexit 0" >> $DAEMON

chmod +x $DAEMON

. /lib/lsb/init-functions

case "$1" in
start)
log_daemon_msg "Starting the process" "$DAEMON"
if start-stop-daemon --start --quiet --exec $DAEMON; then
log_end_msg 0
else
log_end_msg 1
fi
;;
*)
## If no parameters are given, print which are avaiable.
echo "Usage: $0 {start}"
exit 1
;;
esac

exit 0

Experimental script to update eMMC from image:

Reveal hidden contents

Experimental script to update SATA SSD from image:

Reveal hidden contents

Whole customization process with more details and test results can be found on Armbian Customization thread.

Latest prototype insides and test results can be found in the attached files.

Any ideas, comments and references are welcome.

Best regards

Chris

Lime2-rev-C-E-comp-06-p8.pdfFetching info...

Lime2-rev-C-E-comp-06-7p.pdfFetching info...

neomanic · July 15, 2016

Thank you for your work on this, as per my comments in the customisation and 'debian on lime2-emmc' threads. I have my system up and working the way I like it, but it is not automated in the fashion you are doing here... I would very much like it to be, but it is low on my priority list as we have to deploy this hardware very soon. Hopefully after that I can be more useful, but I am only just now getting back to embedded Linux for the first time since 2007.

Back to the matter at hand. I have a few questions for my own understanding.

The u-boot and kernel patches are the same as the ones in the current Armbian release, correct? Is there a reason why the lime2-emmc hasn't been defined as a separate board so we can eventually have it mainlined?

With the boot order patch, why do it this way rather than customising the boot.cmd?

FYI, I changed my boot.cmd to the following which boots from the SATA drive if found, if not then continues with the eMMC. If a bootable SD card is inserted, it overrides both of them.

I see that your boot script is more robust because it tries loading the kernel image, fdt, and initramfs and falls over to another boot device if those fail, whereas mine will require hardware removal if those fail. I will be changing mine to be more like yours in that regard.

boot.cmd

Reveal hidden contents

(p.s. what does that setup control? Looks very nice.)

zador.blood.stained · July 15, 2016

On 7/14/2016 at 9:59 PM, chradev said:

Not solved issues are:

to mount root accordingly without explicit setting in /etc/fstab;

You don't have to have a rootfs entry in fstab - just pass "rw" and "rootflags" to kernel command line to mount it from kernel

setenv bootargs "root=${uuid} rw rootflags=noatime,nodiratime ..."

On 7/15/2016 at 1:47 AM, neomanic said:
The u-boot and kernel patches are the same as the ones in the current Armbian release, correct? Is there a reason why the lime2-emmc hasn't been defined as a separate board so we can eventually have it mainlined?

lime2-emmc is mainlined - separate DT file will be available in kernel 4.7

Edited July 16, 2016 by zador.blood.stained
Copypaste gone wrong: rootargs->rootflags

wildcat_paris · July 15, 2016

On 7/15/2016 at 9:58 AM, zador.blood.stained said:
setenv bootargs "root=${uuid} rw rootargs=noatime,nodiratime ..."

Oh! I am reading "uuid"... cool!

chradev · July 16, 2016

Hi Neimatic,

On 7/15/2016 at 1:47 AM, neomanic said:

The u-boot and kernel patches are the same as the ones in the current Armbian release, correct? Is there a reason why the lime2-emmc hasn't been defined as a separate board so we can eventually have it mainlined?

With the boot order patch, why do it this way rather than customising the boot.cmd?

...

(p.s. what does that setup control? Looks very nice.)

U-Boot and Kernel patches I have used are the same as in Armbian release where they are disabled.

I have already offered to add new Lime2-eMMC board in Armbian but probably this will be happened after adopting Kernel 4.7 where lime2-emmc will be mainlined in separate DT file as noted by Zador.

Boot order patch (or overwriting it with uEnv.txt) is better solution permitting boot.cmd to be kept unified, simple and robust as you mention.

If you mention "setup control" related to the user button at power on forcing boot from eMMC it is an idea used in embedded devices and routers to load factory default FW when something goes wrong.

@Zador,

Thanks for your notes on how to add root flags to kernel line and coming new Lime2-eMMC support in kernel 4.7.

Best regards

Chris

neomanic · July 19, 2016

Yes, thank you @Zador! That answers my question.

@chradev: thank you for the answers. Re: my question, it was about your system which you answered via PM.

chradev · July 21, 2016

Thanks Zador,

On 7/15/2016 at 9:58 AM, zador.blood.stained said:
You don't have to have a rootfs entry in fstab - just pass "rw" and "rootflags" to kernel command line to mount it from kernel
setenv bootargs "root=${uuid} rw rootflags=noatime,nodiratime ..."
lime2-emmc is mainlined - separate DT file will be available in kernel 4.7

It works perfect and makes possible to specify from the boot script both RO FS for eMMC and RW FS for SD and SSD in my case.

EDIT: Unfortunately, it works because I have used wrong rootargs instead of rootflags so kernel ignored mount options.

But after moving to right rootflags kernel crashes with followint error:

[    5.129646] EXT4-fs (mmcblk0p1): Unrecognized mount option "noatime" or missing value
[    5.138951] EXT4-fs (mmcblk0p1): Unrecognized mount option "noatime" or missing value
[    5.146924] List of all partitions:
[    5.150429] 0100            4096 ram0  (driver?)
[    5.155088] 0101            4096 ram1  (driver?)
[    5.159717] 0102            4096 ram2  (driver?)
[    5.164343] 0103            4096 ram3  (driver?)
[    5.168995] b300         3776512 mmcblk1  driver: mmcblk
[    5.174317]   b301         3582976 mmcblk1p1 16072004-01
[    5.179655] b310           16384 mmcblk1boot1  (driver?)
[    5.185003] b308           16384 mmcblk1boot0  (driver?)
[    5.190322] b318         3887104 mmcblk0  driver: mmcblk
[    5.195662]   b319         1547264 mmcblk0p1 000d62f5-01
[    5.200978] No filesystem could mount root, tried:  ext4
[    5.206328] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(179,25)

The flags sent from U-Boot to Kernel via command line are:

setenv extended_bootargs root=PARTUUID=${uuid} rootfstype=ext4 rw rootflags=noatime,nodiratime,commit=600,errors=remount-ro rootwait
setenv bootargs console=${console} ${extended_bootargs} panic=10 loglevel=${verbosity}

Any ideas about what is wrong?

Best regards

Chris

chradev · July 22, 2016

Hi to All,

On 7/21/2016 at 1:00 PM, chradev said:

Thanks Zador,

It works perfect and makes possible to specify from the boot script both RO FS for eMMC and RW FS for SD and SSD in my case.

EDIT: Unfortunately, it works because I have used wrong rootargs instead of rootflags so kernel ignored mount options.

But after moving to right rootflags kernel crashes with followint error:

[    5.129646] EXT4-fs (mmcblk0p1): Unrecognized mount option "noatime" or missing value
[    5.138951] EXT4-fs (mmcblk0p1): Unrecognized mount option "noatime" or missing value
[    5.146924] List of all partitions:
[    5.150429] 0100            4096 ram0  (driver?)
[    5.155088] 0101            4096 ram1  (driver?)
[    5.159717] 0102            4096 ram2  (driver?)
[    5.164343] 0103            4096 ram3  (driver?)
[    5.168995] b300         3776512 mmcblk1  driver: mmcblk
[    5.174317]   b301         3582976 mmcblk1p1 16072004-01
[    5.179655] b310           16384 mmcblk1boot1  (driver?)
[    5.185003] b308           16384 mmcblk1boot0  (driver?)
[    5.190322] b318         3887104 mmcblk0  driver: mmcblk
[    5.195662]   b319         1547264 mmcblk0p1 000d62f5-01
[    5.200978] No filesystem could mount root, tried:  ext4
[    5.206328] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(179,25)

The flags sent from U-Boot to Kernel via command line are:

setenv extended_bootargs root=PARTUUID=${uuid} rootfstype=ext4 rw rootflags=noatime,nodiratime,commit=600,errors=remount-ro rootwait
setenv bootargs console=${console} ${extended_bootargs} panic=10 loglevel=${verbosity}

Any ideas about what is wrong?

Best regards

Chris

After reading of many posts on Internet I find that some options like 'noatime' in 'rootflags' sent via Kernel command line can cause errors.

On the other hand 'noatime' is not a valid mount option for 'ext4' fs.

That is why I leave Kernel command line as:

console=ttyS0,115200 root=PARTUUID=01c14ee2-01 rootfstype=ext4 rw rootflags=discard,errors=remount-ro rootwait panic=10 loglevel=1

and booting from SSD is OK. 'dmesg' says:

EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: discard,errors=remount-ro

which is as expected. Booting from eMMC did not cause any problems as well.

Note 'discard' mount option is recommended for SSD drives. That is why I added it to mount options for the second SSD partition.

Best regards

Chris

chradev · July 26, 2016

Hi to All,

I have decided to publish my posts in PM with Neomatic (Josh) answering his questions noted above in this tread.

Sent 17 July 2016 - 03:03 AM

Reveal hidden contents

Hi Neomatic,

My effort is really targeted to find controllable way to build reproducible, highly customized, embed-able Linux system.

The main requirements are the system to be extremely reliable, fully autonomous and usable from non-qualified users.
It means that the system will never be managed and administered by experts and / or remotely after manufacturing.
The application web interface will be the only user interface accessible via the built-in access point and Ethernet port.
The system will be controlled by the connected to it tablet, smart phone, laptop, etc. devices via web browser only.

Only a power on/off button and a few LEDs are planed for the front panel of the final product.
The only one additional button will be accessible via small hole to restore factory default FW.
There will be no users, console, ssh access even root user will be disabled.

The system will also contains STM32F405 and ChibiOS based part for the real time tasks.
It will be controlled by main application staff and updated by Linux device FW.

There will be 3 completely equivalent systems:

eMMC resided - none update-able and used as fail over,
SSD resided - update-able on the field and used as work
USB resided - used for FW update.

In current customization the system contains a big amount of development staff (incl. RPI Monitor) needed in the development process.
All this staff will be removed in the final product's FW minimizing failure sources and saving resources.

As I suppose your use case is slightly different and more like the general purpose computers.
Fortunately, most of the effort done and described by me could be applicable in your case.
Of course, the administrative web interface could be easily added especially in presence on ngnix, php, node.js etc. packages.

One more use case is planned by us in near future - application servers for FW version control, build and update, CRM and main web site.
Probably this use case will be much closer to your one but it is with lower priority for us by now.

That is way I will be very happy to comment and collaborate on such a project.
In case of interest I can publish my current project (without the application staff) as open source.

Best regards
Chris

Sent 20 July 2016 - 04:15 PM

Reveal hidden contents

Hi Josh,

We use STM32F405 based low level controller for real-time tasks. It is connected via Fast Speed (12Mbps) or optionally High Speed (480Mbps) USB CDC to Lime2-eMMC board. STM32 FW is written on C and is ChibiOS based. For future use external ULPI PHY (USB3300) and ST STM32CubeMX, STM32Cube FW and FreeRTOS were tested to interconnect STM32 and Lime2 parts over HS USB CDC. At cited tests USB Host and Device modes ware tested as well and taking into account that STM32F4 has 2 separate USB ports daisy chaining can be implemented over USB interconnections. Using this opportunity many STM32 MCUs and a single Lime2 can be interconnected as truly distributed system. BTW in case of lower transfer speeds (up to 1Mbps) CAN interface is also an option because both STM32F4 and A20 contain it.

Our STM32 application is mainly used to capture data by built-in ADC using DMA and send them via USB CDC to Lime2 resided server application implemented in JavaScript and running by node.js. In addition a wide set of commands is implemented to control the main process. Top level web application is HTML5 based, written on JavaScript and is running in any browser and any OS, desktop, laptop (Chromebook is very nice option btw), tablet, smartphone etc. connected to Lime2 server via WiFi or Ethernet. For transferring of data between the server and client applications the Web-Socket technology is used. Other advanced web technologies are used as well.

Optional USB-LAN adapter can be used as WAN port to connect the system with Intranet and/or Internet. In this case Lime2 will act as a router for the devices connected to its WiFi and Ethernet interfaces.

Main data transfer rates starts from 150 kBps and can go up to 1MBps over USB and tens of MBps over the GBit LAN and WiFi in case of many clients connected. In addition all data captured can be archived on SATA SSD for a long periods of time (hours). The whole system consisting of wide band ground penetrating radar, STM32 and Lime2 parts has to run many hours from a single battery on the field at wide range of temperatures, humidity and dust.

In addition to above requirements the system has to be easily used by people with no or very little computer knowledge. That is why "one button front panel" and a single web user interfaces will be used to operate the system.

The FW of all its parts has to be upgraded autonomously without expert's assistance. That is way a special care was taken to grant full control over STM32 from SW services running on Lime2. To make it reliably Lime2 USB OTG is used in host mode to connect STM32 and VBUS power is disabled for being able to control it via user services. In such a case STM32 board can be reset at any time. In addition one of the Lime2 GPIOs is used to set STM32 BOOT1 input through reset cycle to put it in boot loaders mode. STM32 internal boot loader is used to upgrade its FW via the same USB connection.

One of the important requirements to be able to recover device after any failure is granted by using read only root FS on eMMC. For running Linux from eMMC fail over mechanism is implemented in U-Boot boot script. In addition a hidden button can also be used to force booting from eMMC. In normal case the system will be booted from SATA SSD with root FS in read write mode. Booting from SD card is preserved for production and repairing only purposes at the factory.

Armbian is used as very good for customization build system. Our custom build is highly customized Debian Jessi image fully configured for none manageable FW update. The main target is to have a controllable, reproducible and reliable way to build image with scripts upgraded from both main Armbian repository and our one.

Best regards

Chris

Best regards

Chris

Sign In

Armbian boot process unification

Recommended Posts

chradev

neomanic

zador.blood.stained

wildcat_paris

chradev

neomanic

chradev

chradev

chradev

Similar Content

Forums

My Activity Streams

Download

Store

Important Information